WO2025112031A1 - 编解码方法、编解码器以及存储介质 - Google Patents

编解码方法、编解码器以及存储介质 Download PDF

Info

Publication number
WO2025112031A1
WO2025112031A1 PCT/CN2023/135808 CN2023135808W WO2025112031A1 WO 2025112031 A1 WO2025112031 A1 WO 2025112031A1 CN 2023135808 W CN2023135808 W CN 2023135808W WO 2025112031 A1 WO2025112031 A1 WO 2025112031A1
Authority
WO
WIPO (PCT)
Prior art keywords
size
block
residual block
current block
residual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2023/135808
Other languages
English (en)
French (fr)
Inventor
王凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to PCT/CN2023/135808 priority Critical patent/WO2025112031A1/zh
Publication of WO2025112031A1 publication Critical patent/WO2025112031A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking

Definitions

  • the present application relates to the field of video coding and decoding technology, and in particular to a coding and decoding method, a codec and a storage medium.
  • the present application provides a coding and decoding method, a codec and a storage medium.
  • the following is an introduction to various aspects of the present application.
  • a decoding method which is applied to a decoder, and the decoding method includes: performing an inverse transformation on the transformation coefficients to determine a first residual block corresponding to a current block, the size of the first residual block being smaller than the size of the current block; determining a second residual block based on the first residual block, the size of the second residual block being equal to the size of the current block; and determining a reconstructed block corresponding to the current block based on the second residual block.
  • a coding method which is applied to an encoder, and the coding method includes: determining a second residual block corresponding to a current block, the size of the second residual block is equal to the size of the current block; determining a first residual block based on the second residual block, the size of the first residual block is smaller than the size of the current block; and transforming the first residual block.
  • a decoder including: a first determination unit, configured to perform an inverse transformation on the transformation coefficients, determine a first residual block corresponding to the current block, and the size of the first residual block is smaller than the size of the current block; a second determination unit, configured to determine a second residual block based on the first residual block, and the size of the second residual block is equal to the size of the current block; and a third determination unit, configured to determine a reconstructed block corresponding to the current block based on the second residual block.
  • a decoder comprising: a memory for storing a computer program; and a processor for executing the method according to the first aspect when running the computer program.
  • an encoder comprising: a first determination unit, configured to determine a second residual block corresponding to a current block, the size of the second residual block being equal to the size of the current block; a second determination unit, configured to determine a first residual block based on the second residual block, the size of the first residual block being smaller than the size of the current block; and a transformation unit, configured to transform the first residual block.
  • an encoder comprising: a memory for storing a computer program; and a processor for executing the method of the second aspect when running the computer program.
  • a computer-readable storage medium stores a computer program, and when the computer program is executed, the method of the first aspect or the second aspect is implemented.
  • a computer program product comprising a computer program, which implements the method of the first aspect or the second aspect when the computer program is executed.
  • a non-volatile computer-readable storage medium for storing a bit stream, wherein the bit stream is generated by an encoding method using an encoder, or the bit stream is decoded by a decoding method using a decoder, wherein the decoding method is the method of the first aspect and the encoding method is the method of the second aspect.
  • a code stream comprising a code stream generated according to the method of the first aspect or a code stream generated according to the method of the second aspect.
  • the embodiment of the present application transforms or inversely transforms blocks of smaller sizes, which helps to reduce the computational complexity of the transform or inversely transform process.
  • FIG. 1 is a diagram showing an example structure of a video encoder to which an embodiment of the present application can be applied.
  • FIG. 2 is a diagram showing an example structure of a video decoder to which an embodiment of the present application can be applied.
  • FIG. 3 is an example diagram of an intra-frame prediction mode in the related art.
  • FIG. 4A is an image that has not been transformed.
  • FIG4B is an image obtained after the image shown in FIG4A is subjected to discrete cosine transform (DCT).
  • DCT discrete cosine transform
  • Figure 5 is the basis image of DCT2.
  • FIG. 6A is a schematic diagram of a coding and decoding process in which only one transformation is performed.
  • FIG. 6B is a schematic diagram of a coding and decoding process based on basic transformation and secondary transformation.
  • FIG. 7 is a diagram showing an example of input and output data of a low frequency non-separable transform (LFNST).
  • LNNST low frequency non-separable transform
  • FIG8 is an example diagram of a transformation kernel group of LFNST.
  • FIG9 is an example diagram of a base image of a non-separable primary transform (NSPT).
  • NSPT non-separable primary transform
  • FIG10 is a flow chart of a decoding method provided in an embodiment of the present application.
  • FIG11 is a flow chart of the encoding method provided in an embodiment of the present application.
  • FIG12 is a schematic diagram of the up/down sampling process of the residual block provided by the implementation of the present application.
  • FIG. 13 is a schematic diagram of the structure of a decoder provided in one embodiment of the present application.
  • FIG14 is a schematic diagram of the structure of a decoder provided in another embodiment of the present application.
  • FIG. 15 is a schematic diagram of the structure of an encoder provided in one embodiment of the present application.
  • FIG16 is a schematic diagram of the structure of an encoder provided in another embodiment of the present application.
  • FIG. 1 is a schematic block diagram of a video encoder according to an embodiment of the present application.
  • the video encoder 100 can be used to perform lossy compression on an image, or can be used to perform lossless compression on an image.
  • the lossless compression can be visually lossless compression or mathematically lossless compression.
  • the video encoder 100 can be applied to image data in a luminance and chrominance (YCbCr, YUV) format.
  • YCbCr, YUV luminance and chrominance
  • the YUV ratio can be 4:2:0, 4:2:2 or 4:4:4, Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) for describing color and saturation.
  • 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr)
  • 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr)
  • 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).
  • the video encoder 100 reads video data, and for each image in the video data, divides an image into a number of coding tree units (CTUs).
  • CTU coding tree units
  • a CTU may be referred to as a "tree block", “largest coding unit” (LCU) or “coding tree block” (CTB).
  • LCU largest coding unit
  • CTB coding tree block
  • Each CTU may be associated with a pixel block of equal size within an image.
  • Each pixel may correspond to a luminance (luminance or luma) sample and two chrominance (chrominance or chroma) samples. Therefore, each CTU may be associated with a luminance sample block and two chrominance sample blocks.
  • a CTU size may be, for example, 128 ⁇ 128, 64 ⁇ 64, 32 ⁇ 32, etc.
  • a CTU may be further divided into a number of coding units (CUs) for encoding, and a CU may be a rectangular block or
  • the video encoder 100 may include: a prediction module 110, a residual module 120, a transform/quantization module 130, an inverse transform/quantization module 140, a reconstruction module 150, a loop filter module 160, a decoded image buffer 170, and an entropy coding module 180. It should be noted that the video encoder 100 may include more, fewer, or different functional components.
  • the current block may be referred to as the current coding unit (CU).
  • the prediction block may also be referred to as a prediction image block or an image prediction block
  • the reconstructed image block may also be referred to as a reconstructed block or an image reconstruction block.
  • an image may be divided into slices. Slices in the same image may be processed in parallel, that is, there is no data dependency between them.
  • "Frame" is a commonly used term, and it can generally be understood that a frame is an image. The frame described in this article may also be replaced by an image or a slice, etc.
  • the prediction module 110 includes an inter-frame prediction module 111 and an intra-frame prediction module 112. Since there is a strong correlation between adjacent pixels in an image of a video, the intra-frame prediction method is used in the video coding and decoding technology to eliminate the spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent images in a video, the inter-frame prediction method is used in the video coding and decoding technology to eliminate the temporal redundancy between adjacent images, thereby improving the coding efficiency.
  • the inter-frame prediction module 111 can be used for inter-frame prediction.
  • Inter-frame prediction can include motion estimation and motion compensation. It can refer to the image information of different images.
  • Inter-frame prediction uses motion information to find a reference block from a reference image, and generates a prediction block based on the reference block to eliminate temporal redundancy.
  • Inter-frame prediction uses motion information to find a reference block from a reference image, and generates a prediction block based on the reference block.
  • the motion information includes a reference image list where the reference image is located, a reference image index, and a motion vector.
  • the motion vector can be an integer pixel or a sub-pixel. If the motion vector is a sub-pixel, it is necessary to use interpolation filtering in the reference image to make the required sub-pixel block.
  • the integer pixel or sub-pixel block in the reference image found according to the motion vector is called a reference block.
  • Some technologies will directly use the reference block as a prediction block, and some technologies will generate a prediction block based on the reference block. Reprocessing the prediction block based on the reference block can also be understood as using the reference block as a prediction block and then processing the prediction block to generate a new prediction block.
  • the intra-frame prediction module 112 only refers to information of the same image to predict pixel information within the current code image block to eliminate spatial redundancy.
  • the intra-frame prediction modes used by high-efficiency video coding include planar mode, direct current mode (DC) and 33 angle modes, for a total of 35 prediction modes.
  • the intra-frame prediction modes used by versatile video coding (VVC) include planar, DC and 65 angle modes, for a total of 67 prediction modes.
  • the residual module 120 may generate a residual block of the CU based on the pixel block of the CU and the prediction block of the CU. For example, the residual module 120 may generate a residual block of the CU so that each sample in the residual block has a value equal to the difference between the following two: a sample in the pixel block of the CU and a corresponding sample in the prediction block of the CU.
  • the transform/quantization module 130 may quantize the transform coefficients.
  • the transform/quantization module 130 may quantize the transform coefficients associated with the CU based on a quantization parameter (QP) value associated with the CU.
  • QP quantization parameter
  • the video encoder 100 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.
  • the inverse transform/quantization module 140 may apply inverse quantization and inverse transform to the quantized transform coefficients, respectively, to reconstruct a residual block from the quantized transform coefficients.
  • the reconstruction module 150 may add the samples of the reconstructed residual block to the corresponding samples of one or more prediction blocks generated by the prediction module 110 to generate a reconstructed image block associated with the CU. By reconstructing each sample block of the CU in this way, the video encoder 100 may reconstruct the pixel block of the CU.
  • the loop filter module 160 is used to process the inverse transformed and inverse quantized pixels to compensate for distortion information and provide a better reference for subsequent encoding pixels. For example, a deblocking filter operation may be performed to reduce the blocking effect of pixel blocks associated with the CU.
  • the loop filter module 160 includes a deblocking filter module and a sample adaptive offset/adaptive loop filter (SAO/ALF) module, wherein the deblocking filter module is used to remove the block effect, and the SAO/ALF module is used to remove the ringing effect.
  • SAO/ALF sample adaptive offset/adaptive loop filter
  • the decoded image buffer 170 may store the reconstructed pixel blocks.
  • the inter prediction module 111 may use the reference image containing the reconstructed pixel blocks to perform inter prediction on PUs of other images.
  • the intra prediction module 112 may use the reconstructed pixel blocks in the decoded image buffer 170 to perform intra prediction on other PUs in the same image as the CU.
  • the entropy encoding module 180 may receive the quantized transform coefficients from the transform/quantization module 130. The entropy encoding module 180 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy-encoded data.
  • FIG. 2 is a schematic block diagram of a video decoder according to an embodiment of the present application.
  • the video decoder 200 includes an entropy decoding module 210, a prediction module 220, an inverse quantization/transformation module 230, a reconstruction module 240, a loop filter module 250, and a decoded image buffer 260. It should be noted that the video decoder 200 may include more, fewer, or different functional components.
  • the video decoder 200 may receive a bitstream.
  • the entropy decoding module 210 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding module 210 may parse the syntax elements in the bitstream that have been entropy encoded.
  • the prediction module 220, the inverse quantization/transformation module 230, the reconstruction module 240, and the loop filter module 250 may decode the video data according to the syntax elements extracted from the bitstream, i.e., generate decoded video data.
  • the prediction module 220 includes an intra-frame prediction module 222 and an inter-frame prediction module 221 .
  • the intra prediction module 222 may perform intra prediction to generate a prediction block of a PU.
  • the intra prediction module 222 may use an intra prediction mode to generate a prediction block of a PU based on pixel blocks of spatially neighboring PUs.
  • the intra prediction module 222 may also determine an intra prediction mode of a PU according to one or more syntax elements parsed from a code stream.
  • the inter prediction module 221 may construct a first reference image list (list 0) and a second reference image list (list 1) according to the syntax elements parsed from the code stream.
  • the entropy decoding module 210 may parse the motion information of the PU.
  • the inter prediction module 221 may determine one or more reference blocks of the PU according to the motion information of the PU.
  • the inter prediction module 221 may generate a prediction block of the PU according to one or more reference blocks of the PU.
  • the inverse quantization/transform module 230 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU.
  • the inverse quantization/transform module 230 may use the QP value associated with the CU of the TU to determine the degree of quantization.
  • inverse quantization/transform module 230 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.
  • the reconstruction module 240 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction module 240 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.
  • the loop filtering module 250 may perform a deblocking filtering operation to reduce blocking effects of pixel blocks associated with a CU.
  • the video decoder 200 may store the reconstructed image of the CU in the decoded image buffer 260.
  • the video decoder 200 may use the reconstructed image in the decoded image buffer 260 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.
  • the basic process of video encoding and decoding is as follows: At the encoding end, an image is divided into blocks.
  • the prediction module 110 uses intra-frame prediction or inter-frame prediction to generate a prediction block of the current block.
  • the residual module 120 can calculate the residual block based on the prediction block and the original block of the current block, that is, The difference between the original block of the predicted block and the current block, the residual block can also be called residual information.
  • the residual block can remove information that is not sensitive to the human eye through processes such as transformation and quantization by the transform/quantization module 130 to eliminate visual redundancy.
  • the residual block before transformation and quantization by the transform/quantization module 130 can be called a time domain residual block
  • the time domain residual block after transformation and quantization by the transform/quantization module 130 can be called a frequency residual block or a frequency domain residual block.
  • the entropy coding module 180 receives the quantized change coefficient output by the transform and quantization module 130, and can perform entropy coding on the quantized change coefficient and output a code stream. For example, the entropy coding module 180 can eliminate character redundancy based on the target context model and the probability information of the binary code stream.
  • the entropy decoding module 210 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block.
  • the prediction module 220 uses intra-frame prediction or inter-frame prediction to generate a prediction block of the current block based on the prediction information.
  • the inverse quantization/transformation module 230 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block.
  • the reconstruction module 240 adds the prediction block and the residual block to obtain a reconstructed block.
  • the reconstructed blocks constitute a reconstructed image, and the loop filtering module 250 performs loop filtering on the reconstructed image based on the image or on the block to obtain a decoded image.
  • the encoding end also requires similar operations as the decoding end to obtain a decoded image.
  • the decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference image for inter-frame prediction for subsequent
  • the block division information determined by the encoder as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., are carried in the bitstream when necessary.
  • the decoder parses the bitstream and determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder by analyzing the existing information, thereby ensuring that the decoded image obtained by the encoder is the same as the decoded image obtained by the decoder.
  • the above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.
  • prediction when encoding, prediction is usually performed first.
  • the so-called prediction refers to obtaining a prediction block that is the same or similar to the current block based on the spatial or temporal correlation of the image.
  • prediction methods such as intra-frame prediction and inter-frame prediction.
  • Intra-frame prediction can also include multiple prediction modes, such as angle prediction mode, etc.
  • the following introduces an intra-frame prediction mode in the related technology, namely matrix-based intra-frame prediction (matrix-based intra prediction, MIP).
  • MIP matrix-weighted intra-frame prediction
  • MIP matrix-weighted intra-frame prediction
  • MIP in order to predict a block with a width of W and a height of H, MIP requires H reconstructed pixels in a column on the left side of the current block and W reconstructed pixels in a row on the upper side of the current block as input.
  • MIP can generate a prediction block in the following three steps: reference pixel averaging, matrix multiplication (matrix vector multiplication) and interpolation.
  • MIP can be understood as the process of generating a prediction block based on input pixels (reference pixels) by matrix multiplication.
  • MIP provides a variety of matrices, and the difference in MIP prediction methods is reflected in the different matrices used. In other words, in MIP, the same input pixel will get different results using different matrices.
  • the process of reference pixel averaging and interpolation is a design that compromises performance and complexity.
  • an effect similar to downsampling can be achieved by averaging reference pixels, so that the input can be adapted to a smaller matrix.
  • Interpolation achieves an upsampling effect.
  • MIP matrices for coding blocks of each size, but only one or several matrices of specific sizes are provided.
  • MIP and planar mode types but obviously MIP is more complex and more flexible than planar.
  • the hybrid coding framework subtracts the predicted block from the original block of the current block to obtain a residual block.
  • the residual block is usually much simpler than the original block, so the use of prediction can significantly improve the compression efficiency.
  • the encoder usually does not encode the residual block directly, but transforms it first.
  • Transformation is to transform the residual block from the spatial domain to the frequency domain. After the residual block is transformed to the frequency domain, since the energy is mostly concentrated in the low-frequency area, the transformed non-zero coefficients are mostly concentrated in the upper left corner. Next, quantization can be used to further compress these non-zero coefficients. In addition, since the human eye is not sensitive to the high-frequency area, a larger quantization step can be used for the quantization coefficients in the high-frequency area.
  • FIG. 4A is the original image.
  • FIG. 4B is the image obtained after the DCT transformation.
  • FIG. 4B shows that after the original image is transformed by DCT, only the upper left corner area has non-zero coefficients.
  • FIG. 4A and FIG. 4B use the DCT transformation of the entire image as an example for explanation, and in video encoding and decoding, the image is usually processed by segmentation and block, so the transformation is also performed based on blocks.
  • transforms are very useful in video compression. However, not all coded blocks need to be transformed. In some cases, transforming a coded block will lead to poor compression. Therefore, in some standards (such as VVC), the encoder can choose whether to transform the current block.
  • DCT2 is the most commonly used transform in video compression standards. Its transformed base image is shown in Figure 5.
  • the transformations in the current standards all use one-dimensional transformations.
  • the two-dimensional transformation is split into one-dimensional transformations in the horizontal direction and the vertical direction, and is performed in two steps.
  • the horizontal transformation can be performed first and then the vertical transformation.
  • the vertical transformation can be performed first and then the horizontal transformation.
  • multiple transform modes such as DCT2, DCT8 and DST7 are supported.
  • each transform mode multiple transform cores can be included.
  • the transform core can be applied separately to the transform in the horizontal or vertical direction.
  • the encoder can select a suitable transform core and transmit the index of the transform core to the bitstream.
  • the decoder can determine the transform core for the inverse transform based on the index. Different transform cores can be selected for horizontal and vertical transforms. For example, DCT8 can be used for transform in the horizontal direction, and DST7 can be used for transform in the vertical direction.
  • MTS multiple transform selection
  • trTypeHor represents the transform kernel of the horizontal transform
  • trTypeVer represents the transform kernel of the vertical transform.
  • the values of trTypeHor and trTypeVer have the following meanings: 0 represents DCT2 type transform; 1 represents DST7 type transform; 2 represents DCT8 type transform.
  • FIG. 6A shows a normal encoding and decoding process without LFNST.
  • FIG. 6A shows a normal encoding and decoding process without LFNST.
  • FIG. 6A shows only one primary transform is performed during the encoding process. Accordingly, only one inverse transform of the primary transform is performed during the decoding process.
  • FIG. 6B shows the encoding and decoding process after the introduction of LFNST. As shown in FIG. 6B, during the encoding process, before the quantization operation is performed, a transform based on DCT2 is performed first, and then a transform based on LFNST is performed.
  • LFNST is a transform performed again on the basis of a basic transform (such as DCT2), so LFNST is usually called a secondary transform.
  • the basic transform mentioned above may include transform modes such as DCT2, DCT8, and DST7.
  • LFNST is used to perform a secondary transformation on the low-frequency coefficients after the basic transformation.
  • the basic transformation concentrates the energy to the upper left corner of the image by decorrelating the image.
  • LFNST transforms the low-frequency coefficients after the basic transformation again to remove the correlation.
  • Figure 8 shows some base images of LFNST in VVC. As can be seen from Figure 8, there is a That is to say, the transformation effect of the oblique image texture based on the transformation kernel groups 1 to 3 is better. And the transformation effect of the flat gradient image texture based on the transformation kernel group 0 is better.
  • the transform kernel selected by LFNST can be determined based on the intra prediction mode. Specifically, the angle prediction tiles the reference pixels to the current block as the prediction value according to the specified angle, which means that the predicted block will have obvious directional texture. The residual block of the current block after angle prediction will also show obvious angle characteristics statistically. Therefore, the transform kernel selected by LFNST can be bound to the intra prediction mode. That is to say, after determining the intra prediction mode, LFNST can use a set of transform kernels corresponding to the intra prediction mode.
  • the LFNST in VVC has 4 groups of transform kernels, each group includes 2 transform kernels.
  • Table 3 shows the correspondence between intra prediction modes and transform kernel groups. It should be understood that in this table, prediction modes 81 to 83 are cross-component prediction modes used when chroma blocks are intra-predicted, while these prediction modes are not used when luminance blocks are intra-predicted.
  • the transform kernel of LFNST can be transposed so that more angles can be processed using one transform kernel group.
  • prediction modes 13 to 23 and prediction modes 45 to 55 can both correspond to transform kernel group 2, but prediction modes 13 to 23 are obviously closer to horizontal prediction modes, while prediction modes 45 to 55 are obviously closer to vertical prediction modes.
  • VVC based on the correlation between the intra prediction mode and the transform core of LFNST, it is possible to determine which set of transform cores LFNST uses, thereby reducing the number of transform cores transmitted in the bitstream. In addition, it is possible to determine whether a coding block will use LFNST through the bitstream and some conditions; if LFNST is used, it is possible to further determine whether the coding block uses the first transform core in a transform core group or the second transform core.
  • the transform core group of LFNST is further expanded to support more transform core groups.
  • LFNST in the related art has 35 transform core groups (each transform core group may include 3 transform cores), and the correspondence between the transform core groups and the intra prediction modes is shown in Table 4. Since more transform core groups are introduced, each transform core group will process the corresponding texture more efficiently.
  • the basic transform such as DCT2
  • the secondary transform such as LFNST
  • the basic transform plus the secondary transform can be understood as a compromise between performance and complexity.
  • NSPT can not only process oblique textures, but also does not require multiple transforms. The following is an introduction to the NSTP provided by the related art.
  • NSPT matches the transform core group in the same way as LFNST, that is, the transform core group is matched according to the intra-frame prediction mode.
  • the specific matching method can refer to the LFNST method, that is, each transform core group includes 3 transform cores.
  • An 8 ⁇ 8 base image of NSPT in the related art is shown in Figure 9.
  • Figure 9 corresponds to inter-frame angle prediction mode 7.
  • many base images of NSPT have obvious oblique textures, so NSTP is very effective in processing the oblique texture of the image.
  • NSPT transforms the spatial domain residuals into frequency domain coefficients.
  • the input to the NSPT transformation is all possible residual values.
  • a 4 ⁇ 8 block has 32 pixels, so the NSPT at the encoder has 32 inputs.
  • an 8 ⁇ 16 block has 128 pixels. pixels, then the NSPT at the encoder has 128 inputs.
  • the output of the encoder can be less than the input, which can reduce the upper limit of the number of coefficients written in the bitstream, thereby reducing the amount of calculation in the transformation process and the storage space of the transformation kernel. Of course, this will also cause distortion, but because of the existence of quantization, this can achieve a better cost-effectiveness.
  • NSPT transforms the coefficients in the frequency domain to the residual in the spatial domain.
  • the output of the inverse transform based on NSPT is the residual value.
  • Table 5 is the transform kernel information table of NSPT at the decoding end in the related art. Referring to Table 5, a 4 ⁇ 8 block has 32 pixels, so the NSPT at the decoding end has 32 outputs. An 8 ⁇ 16 block has 128 pixels, so the NSPT at the decoding end has 128 outputs. The input at the decoding end can be less than the output.
  • the number of transform core groups and the number of transform cores in each group corresponding to NSPT are the same as those of LFNST. Of course, this may be changed in future designs, and the embodiments of the present application do not make specific restrictions on this.
  • the number of inputs at the decoding end is determined by the specific transform core design, which may also be changed in future designs, and the embodiments of the present application do not make specific restrictions on this. For example, for a 4 ⁇ 8 block, as shown in Table 5, the current design is 20 inputs, and it can also be designed as 16 inputs in the future.
  • the number of outputs is determined by the size of the block.
  • the NSPT inverse transform process at the decoding end is a matrix multiplication process.
  • the transform kernel is a matrix of (number of inputs ⁇ number of outputs). Therefore, the larger the (number of inputs ⁇ number of outputs), the larger the storage space required for the transform kernel and the greater the amount of calculation.
  • NSPT is only applied to some small-sized current blocks, while large-sized current blocks still use DCT2+LFNST.
  • the small sizes mentioned here include, for example, 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4, 8 ⁇ 32, and 32 ⁇ 8.
  • the following introduces the application of NSPT in the encoding and decoding process.
  • the encoder first generates a residual block, which can be obtained by subtracting the predicted value from the original value of the current block.
  • the residual block is then transformed. If the size of the current block meets the application conditions of NSPT, the current block can use NSPT; otherwise, the current block can use a combination of basic transform plus secondary transform (such as LFNST).
  • the encoder has multiple options for transforming the residual block. For example, only basic transforms (such as DCT2, DST7, and DCT8, etc.) can be used. Alternatively, NSPT or a combination of basic transform plus secondary transform (such as LFNST) can be used according to the size of the current block.
  • NSPT can be used for block sizes of 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4, 8 ⁇ 32, and 32 ⁇ 8, while DCT2+LFNST can be used for other block sizes.
  • the current block can also skip the transformation process, that is, no transformation is performed.
  • the encoder can select a suitable transformation mode for transformation.
  • the decoder first performs entropy decoding and dequantization to obtain coefficients.
  • the decoder can determine the transform mode based on the indication of the code stream and known information.
  • the selectable transform modes include DCT2, DST7, DCT8, DCT2+LFNST, and NSPT.
  • the decoder can distinguish between NSPT or a combination of a basic transform plus a secondary transform (such as LFNST) based on the block size.
  • NSPT can be used for block sizes of 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4, 8 ⁇ 32, and 32 ⁇ 8, and DCT2+LFNST can be used for other block sizes.
  • NSPT can provide higher compression efficiency than the basic transform (DCT2) + secondary transform (LFNST).
  • DCT2+LFNST basic transform
  • the relevant technology only uses NSPT on small blocks, and still uses DCT2+LFNST on large blocks.
  • an embodiment of the present application provides an encoding method, including: determining a second residual block corresponding to a current block, the size of the second residual block is equal to the size of the current block; determining a first residual block based on the second residual block, the size of the first residual block is smaller than the size of the current block; and transforming the first residual block.
  • the embodiment of the present application further provides a decoding method, comprising: performing an inverse transformation on the transform coefficients to determine a first residual block corresponding to the current block, wherein the size of the first residual block is smaller than the size of the current block; determining a second residual block according to the first residual block, wherein the first residual block The size of the second residual block is equal to the size of the current block; and a reconstructed block corresponding to the current block is determined according to the second residual block.
  • the embodiment of the present application transforms or inversely transforms blocks of smaller sizes, which helps to reduce the computational complexity of the transform or inversely transform process.
  • Fig. 10 is a schematic flow chart of a decoding method provided in an embodiment of the present application. The method in Fig. 10 can be applied to a decoder.
  • the transform coefficients are inversely transformed to determine the first residual block corresponding to the current block.
  • the code stream may be parsed first to determine the transform coefficients; then, the transform coefficients may be inversely transformed to determine the first residual block.
  • the current block may refer to the current block to be decoded.
  • the current block is a luminance block.
  • the current block may also be a chrominance block.
  • the size of the first residual block is smaller than the size of the current block.
  • the size of the first residual block may be associated with the size of the current block.
  • the size of the first residual block may be determined based on the size of the current block.
  • the size of the first residual block may be proportional to the size of the current block. Assuming that the ratio of the width to the height of the first residual block is a first ratio and the ratio of the width to the height of the current block is a second ratio, the first ratio may be equal to the second ratio.
  • the size of the first residual block is 8 ⁇ 8; if the size of the current block is 32 ⁇ 32, the size of the first residual block is 8 ⁇ 8; if the size of the current block is 64 ⁇ 64, the size of the first residual block is 8 ⁇ 8; if the size of the current block is 8 ⁇ 32, the size of the first residual block is 4 ⁇ 16; if the size of the current block is 32 ⁇ 8, the size of the first residual block is 16 ⁇ 4; if the size of the current block is 8 ⁇ 64, the size of the first residual block is 4 ⁇ 32; if the size of the current block is 64 ⁇ 8, the size of the first residual block is 32 ⁇ 4; if the size of the current block is 32 ⁇ 64, the size of the first residual block is 8 ⁇ 16; if the size of the current block is 64 ⁇ 32, the size of the first residual block is 16 ⁇ 8.
  • the second residual block is determined according to the first residual block.
  • the size of the second residual block is equal to the size of the current block. In other words, the size of the second residual block is larger than the size of the first residual block.
  • the second residual block can be determined by upsampling the first residual block.
  • upsampling is first performed in the horizontal direction, and then upsampling is performed in the vertical direction.
  • upsampling is first performed in the vertical direction, and then upsampling is performed in the horizontal direction. Upsampling along a one-dimensional direction can reduce computational complexity.
  • the first residual block can also be upsampled in two dimensions, which helps to improve sampling accuracy.
  • the first residual block can be upsampled based on linear interpolation.
  • the first residual block can be upsampled based on interpolation filtering.
  • a reconstructed block corresponding to the current block is determined according to the second residual block.
  • the current block may be predicted first to determine the predicted block corresponding to the current block.
  • the reconstructed block may be determined according to the second residual block and the predicted block.
  • the method of determining the reconstructed block according to the second residual block and the predicted block may include, for example, determining the sum of the pixels of the second residual block and the predicted block as the reconstructed block.
  • a smaller residual block (smaller in size relative to the current block) is obtained after the inverse transformation operation. That is, the inverse transformation operation is an inverse transformation operation performed on a smaller block, which helps to reduce the computational complexity of the inverse transformation process.
  • the embodiment of the present application does not specifically limit the type of the inverse transform operation performed in step S1010.
  • the embodiment of the present application can be applied to scenarios where the complexity of the inverse transform operation is significantly positively correlated with the size of the block.
  • the NSPT scheme is currently only applied to blocks of smaller size. This is because the larger the size of the block, the greater the computational complexity of the NSPT. Therefore, the embodiment of the present application can be applied to encoding and decoding scenarios based on NSPT, thereby reducing the computational complexity of the transform/inverse transform operation.
  • the embodiments of the present application are applicable to current blocks of any size.
  • the decoding method shown in Figure 10 is used; if the size of the current block does not belong to a specific size or a specific size set, the residual block with the same size as the current block can be directly determined by inverse transformation.
  • a residual block with the same size as the small block can be obtained by inverse transforming the transform coefficients, and the reconstructed block corresponding to the current block is determined based on the residual block;
  • a first residual block with a smaller size than the large block can be determined by inverse transforming the transform coefficients.
  • a second residual block with the same size as the large block is determined based on the first residual block, and the reconstructed block corresponding to the current block is determined based on the second residual block.
  • the transform coefficients can be inversely transformed based on NSPT to determine the first residual block (smaller in size than the current block). Then, the second residual block can be determined based on the first residual block, and the reconstructed block corresponding to the current block can be determined based on the second residual block.
  • the determination method of the second residual block and the reconstructed block is described in Figure 10, which will not be repeated here.
  • the first size set mentioned here can, for example, include at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • a third residual block (with a size equal to the current block) can be determined directly by inverse transformation based on NSPT. Then, the reconstructed block corresponding to the current block can be determined based on the third residual block. For example, if the size of the current block is one of the following sizes, the third residual block can be directly determined by inverse transformation based on NSPT: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • whether the current block belongs to the first size set can be determined based on a preset threshold. For example, if the size of the current block is greater than or equal to the first threshold, the size of the current block belongs to the first size set.
  • the inverse transformation is performed based on NSPT, which can fully utilize the advantage of high compression efficiency of NSPT and improve the decoding performance as a whole.
  • the transform coefficients can be inversely transformed based on NSPT to determine the first residual block (smaller in size than the current block). Then, the second residual block can be determined based on the first residual block, and the reconstructed block corresponding to the current block can be determined based on the second residual block.
  • the determination method of the second residual block and the reconstructed block is described in Figure 10, which will not be repeated here.
  • the first size set mentioned here may include blocks of larger sizes.
  • the first size set may include at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • the third residual block (with a size equal to the current block) can be determined directly by inverse transformation based on NSPT. Then, the reconstructed block corresponding to the current block can be determined according to the third residual block. For example, the reconstructed block can be determined according to the third residual block and the prediction block.
  • the second size set mentioned here may include blocks of relatively small sizes.
  • the second size set may include at least one of the following sizes: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • an inverse transform can be performed based on the basic transform plus secondary transform or the basic transform to determine the fourth residual block (the size is equal to the current block). Then, the reconstructed block corresponding to the current block can be determined according to the fourth residual block. For example, the reconstructed block can be determined according to the fourth residual block and the prediction block.
  • the third size set mentioned here may include, for example, other sizes except the sizes in the first size set and the second size set. For example, the third size set may include 128 ⁇ 64.
  • whether the current block belongs to the first size set can be determined based on a preset threshold. For example, if the size of the current block is greater than or equal to the first threshold, the size of the current block belongs to the first size set.
  • the current block may also be judged based on a preset threshold whether it belongs to the second size set or the third size set. For example, if the size of the current block is less than or equal to the second threshold, the size of the current block belongs to the second size set.
  • implementation method 2 provides more inverse transformation methods, so that they can be flexibly selected according to actual conditions.
  • the second residual block can be determined by upsampling the first residual block.
  • the upsampling method is described in more detail below.
  • the value of the newly interpolated point can be the average value of the values of its two adjacent points. Assuming an upsampling ratio of 1:4, the value of the newly interpolated point can be the weighted average value of the values of its two adjacent original points, and the weight is inversely proportional to the distance, that is, the weights are 3:1, 2:2, and 1:3 respectively.
  • the width before upsampling be widthDwn
  • the width after upsampling be widthUp
  • the parameter xUp widthUp/widthDwn.
  • the residual array before upsampling is resDwn[widthDwn]
  • the residual array after upsampling is resUp[widthUp].
  • the relationship between the upsampling parameters can be expressed by the following formulas (1), (2), and (3).
  • the 8-tap interpolation filter used in VVC for sub-pixel motion vector motion compensation can be reused.
  • a smaller number of taps can be used, such as a 4-tap interpolation filter.
  • the method of FIG10 may further include: parsing a bitstream to determine quantization coefficients; and dequantizing the quantization coefficients to determine transformation coefficients. After obtaining the transformation coefficients, the transformation coefficients may be dequantized to determine a first residual block of the current block.
  • Fig. 11 is a schematic flow chart of an encoding method provided in an embodiment of the present application. The method of Fig. 11 can be applied to an encoder.
  • step S1110 a second residual block corresponding to the current block is determined.
  • the size of the second residual block is equal to the size of the current block.
  • the current block may refer to the current block to be encoded.
  • the current block is a luminance block.
  • the current block may also be a chrominance block.
  • the current block before executing step S1110, may be predicted to determine a prediction block corresponding to the current block; then, the second residual block may be determined based on the prediction block.
  • the method of determining the second residual block based on the prediction block may include, for example, determining the difference between the current block and the prediction block as the second residual block.
  • the first residual block is determined according to the second residual block.
  • the size of the first residual block is smaller than the size of the current block.
  • the size of the second residual block is larger than the size of the first residual block.
  • the first residual block can be determined by downsampling the second residual block.
  • downsampling is first performed in the horizontal direction, and then downsampled in the vertical direction.
  • downsampling is first performed in the vertical direction, and then downsampled in the horizontal direction. Downsampling along a one-dimensional direction can reduce computational complexity.
  • the second residual block can also be downsampled in two dimensions, which helps to improve sampling accuracy.
  • the second residual block can be downsampled based on an average value.
  • the second residual block can be downsampled based on a random value.
  • the size of the first residual block is smaller than the size of the current block.
  • the size of the first residual block may be associated with the size of the current block.
  • the size of the first residual block may be determined based on the size of the current block. Assuming that the ratio of the width to the height of the first residual block is a first ratio and the ratio of the width to the height of the current block is a second ratio, the first ratio may be equal to the second ratio.
  • the size of the first residual block is 8 ⁇ 8; if the size of the current block is 32 ⁇ 32, the size of the first residual block is 8 ⁇ 8; if the size of the current block is 64 ⁇ 64, the size of the first residual block is 8 ⁇ 8; if the size of the current block is 8 ⁇ 32, the size of the first residual block is 4 ⁇ 16; if the size of the current block is 32 ⁇ 8, the size of the first residual block is 16 ⁇ 4; if the size of the current block is 8 ⁇ 64, the size of the first residual block is 4 ⁇ 32; if the size of the current block is 64 ⁇ 8, the size of the first residual block is 32 ⁇ 4; if the size of the current block is 32 ⁇ 64, the size of the first residual block is 8 ⁇ 16; if the size of the current block is 64 ⁇ 32, the size of the first residual block is 16 ⁇ 8.
  • the first residual block is transformed.
  • the first residual block may be transformed to determine the transformation coefficient of the current block.
  • the transformation coefficient may be quantized to determine the quantization coefficient; and the quantization coefficient may be encoded.
  • a residual block of smaller size (relative to the current block) is obtained after step S1120. Therefore, the transformation operation in step S1130 is a transformation operation performed on a block of smaller size, which helps to reduce the computational complexity of the transformation process.
  • the embodiment of the present application does not specifically limit the type of transformation operation performed in step S1130.
  • the embodiment of the present application can be applied to scenarios where the complexity of the transformation operation is significantly positively correlated with the size of the block.
  • the NSPT scheme is currently only applied to blocks of smaller size. This is because the larger the size of the block, the greater the computational complexity of the NSPT. Therefore, the embodiment of the present application can be applied to NSPT-based coding and decoding scenarios to reduce the computational complexity of the transformation/inverse transformation operations.
  • the embodiments of the present application are applicable to current blocks of any size.
  • the encoding method shown in Figure 11 is used; if the size of the current block does not belong to a specific size or a specific size set, the second residual block can be directly transformed to determine the transformation coefficient of the current block.
  • the current block is a small block, a residual block with the same size as the small block can be determined, and the transformation is performed based on the residual block; if the current block is a large block, a second residual block with the same size as the large block can be determined. Then, based on the second residual block, a first residual block smaller than the size of the large block is determined, and the transformation is performed according to the first residual block.
  • Two possible implementations are given below.
  • the first residual block can be transformed based on the NSPT to determine the transform coefficient of the current block. Then, the transform coefficient can be quantized to determine the quantization coefficient; the quantization coefficient is encoded.
  • the first size set mentioned here can include, for example, at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • the second residual block can be directly transformed based on NSPT to determine the transform coefficient of the current block. Then, the transform coefficient can be quantized to determine the quantization coefficient; the quantization coefficient is encoded. For example, if the size of the current block is one of the following sizes, the second residual block can be directly transformed based on NSPT to determine the transform coefficient of the current block: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • whether the current block belongs to the first size set can be determined based on a preset threshold. For example, if the size of the current block is greater than or equal to the first threshold, the size of the current block belongs to the first size set.
  • the transformation is performed based on NSPT, which can fully utilize the advantage of high compression efficiency of NSPT and improve the encoding performance as a whole.
  • the first residual block can be transformed based on NSPT to determine the transformation coefficient of the current block. Then, the transformation coefficient can be quantized to determine the quantization coefficient; the quantization coefficient is encoded.
  • the first size set mentioned here can include at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • the second residual block can be directly transformed based on the NSPT to determine the transform coefficient of the current block. Then, the transform coefficient can be quantized to determine the quantization coefficient; the quantization coefficient is encoded.
  • the second size set mentioned here may include blocks of relatively small sizes.
  • the second size set may include at least one of the following sizes: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • the second residual block can be directly transformed based on the basic transformation plus secondary transformation or the basic transformation to determine the transformation coefficient of the current block. Then, the transformation coefficient can be quantized to determine the quantization coefficient; the quantization coefficient is encoded.
  • the third size set mentioned here may include, for example, other sizes except the sizes in the first size set and the second size set.
  • the third size set may include 128 ⁇ 64.
  • whether the current block belongs to the first size set can be determined based on a preset threshold. For example, if the size of the current block is greater than or equal to the first threshold, the size of the current block belongs to the first size set.
  • the current block may also be judged based on a preset threshold whether it belongs to the second size set or the third size set. For example, if the size of the current block is less than or equal to the second threshold, the size of the current block belongs to the second size set.
  • implementation method 2 provides more transformation methods, so that it can be flexibly selected according to actual conditions.
  • the first residual block can be determined by downsampling the second residual block.
  • the downsampling method is described in more detail below.
  • the second residual block Take downsampling the second residual block based on the average value as an example. If the second residual block is downsampled in one dimension, assuming a 2:1 downsampling, the average value of two adjacent points can be used as the value of a point after downsampling. Assuming a 4:1 downsampling, the average value of four adjacent points can be used as the value of a point after downsampling.
  • the width before downsampling be widthOrg
  • the width after downsampling be widthDwn
  • the parameter xDwn widthOrg/widthDown.
  • the residual value array before downsampling is resOrg[widthOrg]
  • the residual value array after downsampling is resDwn[widthDwn].
  • the relationship between the downsampling parameters can be expressed by the following formula (4).
  • the method of FIG. 11 may further include: quantizing the transform coefficients determined based on the first residual block to determine quantization coefficients; and encoding the quantization coefficients.
  • the encoder generates a residual block.
  • the residual block can be obtained by subtracting the predicted value from the original value of the current block.
  • the size of the residual block is equal to the size of the current block.
  • the size of the current block may be the size of the CU. If the chrominance component is currently being encoded, the size of the current block may be the size of the chrominance component block. If a CU is divided into multiple TUs, the size of the current block may also be the size of the TU.
  • the encoder can have multiple options for transforming the residual block. For example, only use DCT2 or DST7, DCT8, etc. of the basic transform. For another example, NSPT can also be used. Of course, it is also possible to skip the transform, that is, not to perform the transform.
  • the encoder can select a suitable transform method for the transform.
  • NSPT can be directly used to transform the residual block under the block sizes of 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, and 32 ⁇ 4.
  • the residual block can be downsampled to the appropriate block size first, and then the reduced residual block can be transformed by NSPT.
  • both the encoding end and the decoding end involve a block size conversion.
  • the block size to which the transformation is applied or the size of the transformation block of the embodiment of the present application the adapted block size.
  • the encoder downsamples the residual block to the adapted block size before the transformation, and transforms it with the adapted block size, that is, the transform block (tb transform block) size is equal to the adapted block size, and performs subsequent quantization and entropy coding.
  • a corresponding relationship between the size of the current block and the size of the adapted block can be preset.
  • the specific corresponding relationship can be shown in Table 6, for example.
  • the decoder obtains the transform coefficients through entropy decoding and inverse quantization.
  • the transform coefficients are inversely transformed to obtain the residual block.
  • the decoder can determine the transform method based on the instructions of the bitstream and known information.
  • NSPT can be directly used to inverse transform the residual block under the block sizes of 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, and 32 ⁇ 4.
  • the adapted block size can be determined based on Table VI.
  • the transform coefficients are inversely transformed based on NSPT to obtain the residual block of the adapted block size.
  • the residual block of the adapted block size is then upsampled to obtain the residual block of the current block.
  • the residual block and the predicted block are added together to obtain the reconstructed block.
  • the current block of a larger size also uses NSPT, but instead of directly using a larger transform kernel, a small transform kernel is applied to the current block of a large size.
  • a small transform kernel is applied to the current block of a large size.
  • the coefficients are inversely transformed using NSPT to obtain a residual block, and the residual block is upsampled to obtain a residual block of a larger size.
  • the key point is to adapt NSPT using upsampling and downsampling in the spatial domain.
  • the transform kernel originally used for the current block of a small size can also be used on the current block of a large size.
  • upsampling and downsampling are lossy and will lose some high-frequency information, but when performing lossy encoding, NSPT and quantization themselves will also lose some high-frequency information, so the distortion generated in this step is acceptable.
  • the "inverse transformation” of the transform coefficients by the decoding end may also be referred to as “transformation” in the standard text.
  • the “transformation” and “inverse transformation” in the embodiments of the present application correspond to two opposite processes. For example, if the "transformation” converts the numerical values in the spatial domain to the coefficients in the frequency domain, then the “inverse transformation” converts the coefficients in the frequency domain to the numerical values in the spatial domain. If the standard only stipulates decoding, then the "transformation” in the standard text is the decoding part, which refers to the "inverse transformation” in this article.
  • the “inverse transformation” of the transform coefficients by the decoding end may also be referred to as "transformation” in the standard text.
  • Fig. 13 is a schematic diagram of the structure of a decoder provided by an embodiment of the present application.
  • the decoder 1300 includes: a first determining unit 1310 , a second determining unit 1320 , and a third determining unit 1330 .
  • the first determining unit 1310 is configured to perform an inverse transformation on the transformation coefficients to determine a first residual block corresponding to the current block, where a size of the first residual block is smaller than a size of the current block.
  • the second determining unit 1320 is configured to determine a second residual block according to the first residual block, wherein a size of the second residual block is equal to a size of the current block.
  • the third determining unit 1330 determines a reconstructed block corresponding to the current block according to the second residual block.
  • the first determining unit 1310 is configured to: perform an inverse transformation on the transformation coefficients based on a first transformation method, where the first transformation method is an inseparable basic transformation.
  • the decoder 1300 further includes: a fourth determination unit configured to determine whether the size of the current block belongs to a first size set; and the first determination unit 1310 is configured to: determine the first residual block if the size of the current block belongs to the first size set.
  • the decoder 1300 further includes: a fifth determination unit, configured to, if the size of the current block does not belong to the first size set, inversely transform the transform coefficients to determine a third residual block, wherein the size of the third residual block is equal to the size of the current block.
  • a fifth determination unit configured to, if the size of the current block does not belong to the first size set, inversely transform the transform coefficients to determine a third residual block, wherein the size of the third residual block is equal to the size of the current block.
  • the fifth determining unit is configured to: perform an inverse transformation on the transformation coefficient based on a first transformation method, where the first transformation method is an inseparable basic transformation.
  • the decoder 1300 also includes: an inverse transformation unit, configured to: if the size of the current block belongs to a second size set, perform an inverse transformation on the transformation coefficient based on a first transformation method; and/or if the size of the current block belongs to a third size set, perform an inverse transformation on the transformation coefficient based on a second transformation method; wherein the first transformation method is different from the second transformation method.
  • an inverse transformation unit configured to: if the size of the current block belongs to a second size set, perform an inverse transformation on the transformation coefficient based on a first transformation method; and/or if the size of the current block belongs to a third size set, perform an inverse transformation on the transformation coefficient based on a second transformation method; wherein the first transformation method is different from the second transformation method.
  • the first transformation method is an inseparable basic transformation
  • the second transformation method is a combination of a basic transformation and a secondary transformation
  • the first set of sizes includes at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • the second set of sizes includes at least one of the following sizes: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • a size of the first residual block is determined based on a size of the current block.
  • a ratio of a width to a height of the first residual block is a first ratio
  • a ratio of a width to a height of the current block is a second ratio
  • the first ratio is equal to the second ratio
  • the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 32 ⁇ 32, the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 64 ⁇ 64, the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 8 ⁇ 32, the size of the first residual block is 4 ⁇ 16; or, if the size of the current block is 32 ⁇ 8, the size of the first residual block is 16 ⁇ 4; or, if the size of the current block is 8 ⁇ 64, the size of the first residual block is 4 ⁇ 32; or, if the size of the current block is 64 ⁇ 8, the size of the first residual block is 32 ⁇ 4; or, if the size of the current block is 32 ⁇ 64, the size of the first residual block is 8 ⁇ 16; or, if the size of the current block is 64 ⁇ 32, the size of the first residual block is 16 ⁇ 8.
  • the second determining unit 1320 is configured to: upsample the first residual block to determine the second residual block.
  • the second determination unit 1320 is configured to: for the first residual block, first upsample along the horizontal direction, and then upsample along the vertical direction; or, for the first residual block, first upsample along the vertical direction, and then upsample along the horizontal direction; or, for the first residual block, perform two-dimensional upsampling.
  • the decoder 1300 further includes: a sixth determination unit configured to: parse the bitstream to determine the quantization coefficient; and a seventh determination unit configured to: dequantize the quantization coefficient to determine the transform coefficient.
  • the third determination unit 1330 is configured to: predict the current block to determine a prediction block corresponding to the current block; and determine the reconstructed block according to the second residual block and the prediction block.
  • the first determining unit 1310 is configured as: the first residual block is determined based on an inseparable basic transform, and a transform core group of the inseparable basic transform is determined based on an intra-frame prediction mode of the prediction block.
  • a "unit” may be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course, it may be a module, or it may be non-modular.
  • the components in the present embodiment may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
  • an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 1200.
  • the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the decoding method in the first embodiment is implemented.
  • the decoder 1400 may include: a communication interface 1410, a memory 1420 and a processor 1430; each component is coupled together through a bus system 1440. It can be understood that the bus system 1440 is used to achieve connection and communication between these components.
  • the bus system 1440 also includes a power bus, a control bus and a status signal bus.
  • various buses are labeled as bus system 1440 in Figure 14. Among them,
  • the communication interface 1410 is used to receive and send signals when sending and receiving information with other external network elements.
  • the memory 1420 is used to store computer programs.
  • Processor 1430 is used to execute, when running the computer program: inverse transform the transform coefficients to determine a first residual block corresponding to the current block, where the size of the first residual block is smaller than the size of the current block; determine a second residual block based on the first residual block, where the size of the second residual block is equal to the size of the current block; and determine a reconstructed block corresponding to the current block based on the second residual block.
  • the memory 1420 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.
  • the non-volatile memory can be a read-only memory (ROM), a volatile Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), or Flash memory.
  • Volatile memory can be Random Access Memory (RAM), which is used as an external cache.
  • RAM Static RAM
  • DRAM Dynamic RAM
  • SDRAM Synchronous DRAM
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • ESDRAM Enhanced Synchronous Dynamic Random Access Memory
  • SLDRAM Synchronous Link Dynamic Random Access Memory
  • DRRAM Direct Rambus RAM
  • the processor 1430 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1430 or the instruction in the form of software.
  • the above processor 1430 may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • the methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general processor can be a microprocessor or the processor can also be any conventional processor, etc.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed.
  • the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc.
  • the storage medium is located in the memory 1420, and the processor 1430 reads the information in the memory 1420 and completes the steps of the above method in combination with its hardware.
  • the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof.
  • ASIC Application Specific Integrated Circuits
  • DSP Digital Signal Processing
  • DSP Device digital signal processing devices
  • PLD programmable logic devices
  • FPGA field programmable gate array
  • general processors controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof.
  • the technology described in this application can be implemented by a module (such as a process, function, etc.) that performs the functions described in this application.
  • the software code can be stored in a memory and executed by a processor.
  • the memory can be implemented in the processor or outside the processor.
  • the processor 1430 is further configured to execute the decoding method described in the above embodiment when running the computer program.
  • Fig. 14 is a schematic diagram of the structure of an encoder provided by an embodiment of the present application.
  • the encoder 1500 includes: a first determination unit 1510 , a second determination unit 1520 and a transformation unit 1530 .
  • the first determining unit 1510 is configured to determine a second residual block corresponding to the current block, where the size of the second residual block is equal to the size of the current block.
  • the second determining unit 1520 is configured to determine a first residual block according to the second residual block, where a size of the first residual block is smaller than a size of the current block.
  • the transform unit 1530 is configured to transform the first residual block.
  • the transform unit 1530 is configured to: transform the first residual block based on a first transform mode, where the first transform mode is a non-separable basic transform.
  • the encoder 1500 also includes: a third determination unit configured to determine whether the size of the current block belongs to a first size set; the second determination unit 1520 is configured to determine the first residual block based on the second residual block if the size of the current block belongs to the first size set.
  • the encoder 1500 further includes: a first transform unit configured to: if the size of the current block does not belong to the first size set, transform the second residual block.
  • the first transform unit is configured to: transform the second residual block based on a first transform mode, where the first transform mode is an inseparable basic transform.
  • the encoder 1500 also includes: a second transformation unit, configured to: if the size of the current block belongs to a second size set, transform the second residual block based on a first transformation method; and/or if the size of the current block belongs to a third size set, transform the second residual block based on a second transformation method; wherein the first transformation method is different from the second transformation method.
  • a second transformation unit configured to: if the size of the current block belongs to a second size set, transform the second residual block based on a first transformation method; and/or if the size of the current block belongs to a third size set, transform the second residual block based on a second transformation method; wherein the first transformation method is different from the second transformation method.
  • the first transformation method is an inseparable basic transformation
  • the second transformation method is a combination of a basic transformation and a secondary transformation
  • the first set of sizes includes at least one of the following sizes: 32 ⁇ 32, 64 ⁇ 64, 8 ⁇ 32, 32 ⁇ 8, 8 ⁇ 64, 64 ⁇ 8, 32 ⁇ 64, 64 ⁇ 32.
  • the second set of sizes includes at least one of the following sizes: 4 ⁇ 4, 4 ⁇ 8, 8 ⁇ 4, 8 ⁇ 8, 4 ⁇ 16, 16 ⁇ 4, 8 ⁇ 16, 16 ⁇ 8, 4 ⁇ 32, 32 ⁇ 4.
  • a size of the first residual block is determined based on a size of the current block.
  • a ratio of a width to a height of the first residual block is a first ratio
  • a ratio of a width to a height of the current block is a second ratio
  • the first ratio is equal to the second ratio
  • the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 32 ⁇ 32, the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 64 ⁇ 64, the size of the first residual block is 8 ⁇ 8; or, if the size of the current block is 8 ⁇ 32, the size of the first residual block is 4 ⁇ 16; or, if the size of the current block is 32 ⁇ 8, the size of the first residual block is 16 ⁇ 4; or, if the size of the current block is 8 ⁇ 64, the size of the first residual block is 4 ⁇ 32; or, if the size of the current block is 64 ⁇ 8, the size of the first residual block is 32 ⁇ 4; or, if the size of the current block is 32 ⁇ 64, the size of the first residual block is 8 ⁇ 16; or, if the size of the current block is 64 ⁇ 32, the size of the first residual block is 16 ⁇ 8.
  • the second determining unit 1520 is configured to: downsample the second residual block to determine the first residual block.
  • the second determination unit 1520 is configured to: for the first residual block, first downsample along the horizontal direction, and then downsample along the vertical direction; or, for the first residual block, first downsample along the vertical direction, and then downsample along the horizontal direction; or, for the first residual block, perform two-dimensional downsampling.
  • the encoder 1500 further includes: a fourth determination unit configured to quantize the transform coefficient determined based on the first residual block to determine the quantization coefficient; and an encoding unit configured to encode the quantization coefficient.
  • the first determination unit 1510 is configured to: predict the current block to determine a prediction block corresponding to the current block; and determine the second residual block according to the prediction block.
  • the transform unit 1530 is configured to: transform the first residual block based on an inseparable basic transform, and a transform kernel group of the inseparable basic transform is determined based on an intra-frame prediction mode of the prediction block.
  • a "unit” may be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course, it may be a module, or it may be non-modular.
  • the components in the present embodiment may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
  • an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 1500.
  • the computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the decoding method in the first embodiment is implemented.
  • the encoder 1600 may include: a communication interface 1610, a memory 1620 and a processor 1630; each component is coupled together through a bus system 1640. It can be understood that the bus system 1640 is used to realize the connection and communication between these components.
  • the bus system 1640 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 1640 in Figure 16. Among them,
  • the communication interface 1610 is used to receive and send signals when sending and receiving information with other external network elements.
  • the memory 1620 is used to store computer programs.
  • Processor 1630 is used to execute, when running the computer program: determining a second residual block corresponding to the current block, the size of the second residual block is equal to the size of the current block; determining a first residual block according to the second residual block, the size of the first residual block is smaller than the size of the current block; transforming the first residual block.
  • the memory 1620 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.
  • the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
  • the volatile memory can be a random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (EEPROM), and the like.
  • the memory 1620 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
  • the processor 1630 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1630 or the instruction in the form of software.
  • the above processor 1630 may be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • the methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general processor can be a microprocessor or the processor can also be any conventional processor, etc.
  • the steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed.
  • the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc.
  • the storage medium is located in the memory 1620, and the processor 1630 reads the information in the memory 1620 and completes the steps of the above method in combination with its hardware.
  • the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof.
  • ASIC Application Specific Integrated Circuits
  • DSP Digital Signal Processing
  • DSP Device digital signal processing devices
  • PLD programmable logic devices
  • FPGA field programmable gate array
  • general processors controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof.
  • the technology described in this application can be implemented by a module (such as a process, function, etc.) that performs the functions described in this application.
  • the software code can be stored in a memory and executed by a processor.
  • the memory can be implemented in the processor or outside the processor.
  • the processor 1630 is further configured to execute the encoding method described in the above embodiment when running the computer program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请提供了一种编解码方法、编解码器以及存储介质。该解码方法包括：对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；根据所述第一残差块确定第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；根据所述第二残差块确定所述当前块对应的重建块。

Description

编解码方法、编解码器以及存储介质 技术领域
本申请涉及视频编解码技术领域，尤其涉及一种编解码方法、编解码器以及存储介质。
背景技术
当前块的尺寸越大，对应的残差块的尺寸也就越大。在编解码过程中，残差块的尺寸越大，进行变换和反变换时的计算复杂度也会相应地增加。因此，如何降低变换和反变换过程的计算复杂度，是需要解决的问题。
发明内容
本申请提供一种编解码方法、编解码器以及存储介质。下面对本申请涉及的各个方面进行介绍。
第一方面，提供了一种解码方法，该解码方法应用于解码器，该解码方法包括：对变换系数进行反变换，确定当前块对应的第一残差块，第一残差块的尺寸小于当前块的尺寸；根据第一残差块确定第二残差块，第二残差块的尺寸等于当前块的尺寸；根据第二残差块确定当前块对应的重建块。
第二方面，提供了一种编码方法，该编码方法应用于编码器，该编码方法包括：确定当前块对应的第二残差块，第二残差块的尺寸等于当前块的尺寸；根据第二残差块确定第一残差块，第一残差块的尺寸小于当前块的尺寸；对第一残差块进行变换。
第三方面，提供了一种解码器，包括：第一确定单元，配置为对变换系数进行反变换，确定当前块对应的第一残差块，第一残差块的尺寸小于当前块的尺寸；第二确定单元，配置为根据第一残差块确定第二残差块，第二残差块的尺寸等于当前块的尺寸；第三确定单元，配置为根据第二残差块确定当前块对应的重建块。
第四方面，提供了一种解码器，包括：存储器，用于存储计算机程序；处理器，用于在运行计算机程序时，执行如第一方面的方法。
第五方面，提供了一种编码器，包括：第一确定单元，配置为确定当前块对应的第二残差块，第二残差块的尺寸等于当前块的尺寸；第二确定单元，配置为根据第二残差块确定第一残差块，第一残差块的尺寸小于当前块的尺寸；变换单元，配置为对第一残差块进行变换。
第六方面，提供了一种编码器，包括：存储器，用于存储计算机程序；处理器，用于在运行计算机程序时，执行如第二方面的方法。
第七方面，提供了一种计算机可读存储介质，其中，计算机可读存储介质存储有计算机程序，计算机程序被执行时实现如第一方面或者第二方面的方法。
第八方面，提供一种计算机程序产品，包括计算机程序，计算机程序被执行时实现如第一方面或第二方面的方法。
第九方面，提供一种存储比特流的非易失性计算机可读存储介质，比特流通过利用编码器的编码方法而生成，或者，比特流通过利用解码器的解码方法而解码，其中，解码方法为第一方面的方法、编码方法为第二方面的方法。
第十方面，提供一种码流，包括根据第一方面的方法生成的码流或者根据第二方面的方法生成的码流。
本申请实施例在编解码过程中，对尺寸较小的块进行变换或反变换，有助于降低变换或反变换过程的计算复杂度。
附图说明
图1为可应用本申请实施例的视频编码器的结构示例图。
图2为可应用本申请实施例的视频解码器的结构示例图。
图3为相关技术中的一种帧内预测模式示例图。
图4A为一幅未经变换处理的图像。
图4B为图4A所示的图像经过离散余弦变换(discrete cosine transform，DCT)之后得到的图像。
图5为DCT2的基图像。
图6A为仅进行一次变换的编解码流程示意图。
图6B为基于基础变换和二次变换的编解码流程示意图。
图7为低频不可分离的变换(low frequency non-separable transform，LFNST)的输入和输出数据的示例图。
图8为LFNST的变换核组的示例图。
图9为不可分离的基础变换(non-separable primary transform，NSPT)的基图像的示例图。
图10为本申请实施例提供的解码方法的流程示意图。
图11为本申请实施例提供的编码方法的流程示意图。
图12为本申请实施提供的残差块的上/下采样过程的示意图。
图13为本申请一个实施例提供的解码器的结构示意图。
图14为本申请另一实施例提供的解码器的结构示意图。
图15为本申请一个实施例提供的编码器的结构示意图。
图16为本申请另一实施例提供的编码器的结构示意图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
图1是本申请实施例涉及的视频编码器的示意性框图。
应理解，该视频编码器100可用于对图像进行有损压缩(lossy compression)，也可用于对图像进行无损压缩(lossless compression)。该无损压缩可以是视觉无损压缩(visually lossless compression)，也可以是数学无损压缩(mathematically lossless compression)。
该视频编码器100可应用于亮度色度(YCbCr，YUV)格式的图像数据上。例如，YUV比例可以为4:2:0、4:2:2或者4:4:4，Y表示明亮度(Luma)，Cb(U)表示蓝色色度，Cr(V)表示红色色度，U和V表示为色度(Chroma)用于描述色彩及饱和度。例如，在颜色格式上，4:2:0表示每4个像素有4个亮度分量，2个色度分量(YYYYCbCr)，4:2:2表示每4个像素有4个亮度分量，4个色度分量(YYYYCbCrCbCr)，4:4:4表示全像素显示(YYYYCbCrCbCrCbCrCbCr)。
例如，该视频编码器100读取视频数据，针对视频数据中的每个图像，将一个图像划分成若干个编码树单元(coding tree unit，CTU)，在一些例子中，CTU可被称作“树型块”、“最大编码单元”(Largest Coding unit，简称LCU)或“编码树型块”(coding tree block，简称CTB)。每一个CTU可以与图像内的具有相等大小的像素块相关联。每一像素可对应一个亮度(luminance或luma)采样及两个色度(chrominance或chroma)采样。因此，每一个CTU可与一个亮度采样块及两个色度采样块相关联。一个CTU大小例如为128×128、64×64、32×32等。一个CTU又可以继续被划分成若干个编码单元(coding unit，CU)进行编码，CU可以为矩形块也可以为方形块。CU可以对应预测单元(prediction Unit，简称PU)和变换单元(transform unit，简称TU)。
在一些实施例中，如图1所示，该视频编码器100可包括：预测模块110、残差模块120、变换/量化模块130、反变换/量化模块140、重建模块150、环路滤波模块160、解码图像缓存170和熵编码模块180。需要说明的是，视频编码器100可包含更多、更少或不同的功能组件。
可选地，在本申请中，当前块可以称为当前编码单元(CU)。预测块也可称为预测图像块或图像预测块，重建图像块也可称为重建块或图像重建块。由于并行处理的需要，图像可以被划分成片(slice)。同一个图像中的slice可以并行处理，也就是说它们之间没有数据依赖。而“帧”是一种常用的说法，一般可以理解为一帧是一个图像。在本文中所述帧也可以替换为图像或slice等。
在一些实施例中，预测模块110包括帧间预测模块111和帧内预测模块112。由于视频的一个图像中的相邻像素之间存在很强的相关性，在视频编解码技术中使用帧内预测的方法消除相邻像素之间的空间冗余。由于视频中的相邻图像之间存在着很强的相似性，在视频编解码技术中使用帧间预测方法消除相邻图像之间的时间冗余，从而提高编码效率。
帧间预测模块111可用于帧间预测，帧间预测可以包括运动估计(motion estimation)和运动补偿(motion compensation)，可以参考不同图像的图像信息，帧间预测使用运动信息从参考图像中找到参考块，根据参考块生成预测块，用于消除时间冗余。帧间预测使用运动信息从参考图像中找到参考块，根据参考块生成预测块。运动信息包括参考图像所在的参考图像列表，参考图像索引，以及运动矢量。运动矢量可以是整像素的或者是分像素的，如果运动矢量是分像素的，那么需要在参考图像中使用插值滤波做出所需的分像素的块，这里把根据运动矢量找到的参考图像中的整像素或者分像素的块叫参考块。有的技术会直接把参考块作为预测块，有的技术会在参考块的基础上再处理生成预测块。在参考块的基础上再处理生成预测块也可以理解为把参考块作为预测块然后再在预测块的基础上处理生成新的预测块。
帧内预测模块112只参考同一图像的信息，预测当前码图像块内的像素信息，用于消除空间冗余。
帧内预测有多种预测模式，以国际数字视频编码标准H系列为例，H.264/AVC标准有8种角度预测模式和1种非角度预测模式，H.265/HEVC扩展到33种角度预测模式和2种非角度预测模式。(high  efficiency video coding，HEVC)使用的帧内预测模式有平面模式(Planar)、直流模式(DC)和33种角度模式，共35种预测模式。多功能视频编码(versatile video coding，VVC)使用的帧内模式有Planar、DC和65种角度模式，共67种预测模式。
需要说明的是，随着角度模式的增加，帧内预测将会更加精确，也更加符合对高清以及超高清数字视频发展的需求。
残差模块120可基于CU的像素块及CU的预测块来产生CU的残差块。举例来说，残差模块120可产生CU的残差块，使得残差块中的每一采样具有等于以下两者之间的差的值：CU的像素块中的采样，及CU的预测块中的对应采样。
变换/量化模块130可量化变换系数。变换/量化模块130可基于与CU相关联的量化参数(QP)值来量化与CU相关联的变换系数。视频编码器100可通过调整与CU相关联的QP值来调整应用于与CU相关联的变换系数的量化程度。
反变换/量化模块140可分别将逆量化及逆变换应用于量化后的变换系数，以从量化后的变换系数重建残差块。
重建模块150可将重建后的残差块的采样加到预测模块110产生的一个或多个预测块的对应采样，以产生与CU相关联的重建图像块。通过此方式重建CU的每一个采样块，视频编码器100可重建CU的像素块。
环路滤波模块160用于对反变换与反量化后的像素进行处理，弥补失真信息，为后续编码像素提供更好的参考，例如可执行消块滤波操作以减少与CU相关联的像素块的块效应。
在一些实施例中，环路滤波模块160包括去块滤波模块和样点自适应补偿/自适应环路滤波(SAO/ALF)模块，其中去块滤波模块用于去方块效应，SAO/ALF模块元用于去除振铃效应。
解码图像缓存170可存储重建后的像素块。帧间预测模块111可使用含有重建后的像素块的参考图像来对其它图像的PU执行帧间预测。另外，帧内预测模块112可使用解码图像缓存170中的重建后的像素块来对在与CU相同的图像中的其它PU执行帧内预测。
熵编码模块180可接收来自变换/量化模块130的量化后的变换系数。熵编码模块180可对量化后的变换系数执行一个或多个熵编码操作以产生熵编码后的数据。
图2是本申请实施例涉及的视频解码器的示意性框图。
如图2所示，视频解码器200包含：熵解码模块210、预测模块220、反量化/变换模块230、重建模块240、环路滤波模块250及解码图像缓存260。需要说明的是，视频解码器200可包含更多、更少或不同的功能组件。
视频解码器200可接收码流。熵解码模块210可解析码流以从码流提取语法元素。作为解析码流的一部分，熵解码模块210可解析码流中的经熵编码后的语法元素。预测模块220、反量化/变换模块230、重建模块240及环路滤波模块250可根据从码流中提取的语法元素来解码视频数据，即产生解码后的视频数据。
在一些实施例中，预测模块220包括帧内预测模块222和帧间预测模块221。
帧内预测模块222可执行帧内预测以产生PU的预测块。帧内预测模块222可使用帧内预测模式以基于空间相邻PU的像素块来产生PU的预测块。帧内预测模块222还可根据从码流解析的一个或多个语法元素来确定PU的帧内预测模式。
帧间预测模块221可根据从码流解析的语法元素来构造第一参考图像列表(列表0)及第二参考图像列表(列表1)。此外，如果PU使用帧间预测编码，则熵解码模块210可解析PU的运动信息。帧间预测模块221可根据PU的运动信息来确定PU的一个或多个参考块。帧间预测模块221可根据PU的一个或多个参考块来产生PU的预测块。
反量化/变换模块230可逆量化(即，解量化)与TU相关联的变换系数。反量化/变换模块230可使用与TU的CU相关联的QP值来确定量化程度。
在逆量化变换系数之后，反量化/变换模块230可将一个或多个逆变换应用于逆量化变换系数，以便产生与TU相关联的残差块。
重建模块240使用与CU的TU相关联的残差块及CU的PU的预测块以重建CU的像素块。例如，重建模块240可将残差块的采样加到预测块的对应采样以重建CU的像素块，得到重建图像块。
环路滤波模块250可执行消块滤波操作以减少与CU相关联的像素块的块效应。
视频解码器200可将CU的重建图像存储于解码图像缓存260中。视频解码器200可将解码图像缓存260中的重建图像作为参考图像用于后续预测，或者，将重建图像传输给显示装置呈现。
视频编解码的基本流程如下：在编码端，将一个图像划分成块，针对当前块，预测模块110使用帧内预测或帧间预测产生当前块的预测块。残差模块120可基于预测块与当前块的原始块计算残差块，即 预测块和当前块的原始块的差值，该残差块也可称为残差信息。该残差块经由变换/量化模块130变换与量化等过程，可以去除人眼不敏感的信息，以消除视觉冗余。可选的，经过变换/量化模块130变换与量化之前的残差块可称为时域残差块，经过变换/量化模块130变换与量化之后的时域残差块可称为频率残差块或频域残差块。熵编码模块180接收到变换量化模块130输出的量化后的变化系数，可对该量化后的变化系数进行熵编码，输出码流。例如，熵编码模块180可根据目标上下文模型以及二进制码流的概率信息消除字符冗余。
在解码端，熵解码模块210可解析码流得到当前块的预测信息、量化系数矩阵等，预测模块220基于预测信息对当前块使用帧内预测或帧间预测产生当前块的预测块。反量化/变换模块230使用从码流得到的量化系数矩阵，对量化系数矩阵进行反量化、反变换得到残差块。重建模块240将预测块和残差块相加得到重建块。重建块组成重建图像，环路滤波模块250基于图像或基于块对重建图像进行环路滤波，得到解码图像。编码端同样需要和解码端类似的操作获得解码图像。该解码图像也可以称为重建图像，重建图像可以为后续的图像作为帧间预测的参考图像。
需要说明的是，编码端确定的块划分信息，以及预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息等在必要时携带在码流中。解码端通过解析码流及根据已有信息进行分析确定与编码端相同的块划分信息，预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息，从而保证编码端获得的解码图像和解码端获得的解码图像相同。
上述是基于块的混合编码框架下的视频编解码器的基本流程，随着技术的发展，该框架或流程的一些模块或步骤可能会被优化，本申请适用于该基于块的混合编码框架下的视频编解码器的基本流程，但不限于该框架及流程。
前文详细描述了本申请实施例提供的编解码框架。如前文所述，在编码时，通常会先进行预测。所谓预测，指的是基于图像在空间或者时间上的相关性能得到一个与当前块相同或相似的预测块。预测方式有很多种，如帧内预测、帧间预测。帧内预测又可以包括多种预测模式，如角度预测模式等。下面介绍一种相关技术中的帧内预测模式，即基于矩阵的帧内预测(matrix-based intra prediction，MIP)。MIP也可以称为矩阵加权平均的帧内预测(matrix weighted intra prediction，MIP)。如图3所示，为了对一个宽度为W，高度为H的块进行预测，MIP需要当前块左侧一列的H个重建像素和当前块上侧一行的W个重建像素作为输入。MIP可以按如下3个步骤生成预测块：参考像素平均(averaging)、矩阵乘法(matrix vector multiplication)和插值(interpolation)。MIP可以理解为基于输入像素(参考像素)通过矩阵乘法的方式生成预测块的过程。MIP提供了多种矩阵，而MIP预测方式的不同就体现在使用的矩阵的不同上。也就是说，在MIP中，相同的输入像素使用不同的矩阵会得到不同的结果。而参考像素平均和插值的过程是一种性能和复杂度折中的设计。对于尺寸较大的编码块，可以通过参考像素平均来实现一种近似于下采样的效果，使输入能适配到比较小的矩阵。而插值则实现一种上采样的效果。这样就不需要对每一种尺寸的编码块都提供MIP的矩阵，而是只提供一种或几种特定的尺寸的矩阵即可。随着对压缩性能的需求的提高，以及硬件能力的提高，下一代的标准中也许会出现复杂度更高的MIP。MIP与平面(planar)模式类型，但显然MIP比planar更复杂，灵活性也更强。
对一个编码块来说，预测块和原始块完全相同的情况是有可能出现的。但是，很难保证一个视频中的所有编码块的预测块与原始块均完全相同。特别地，对自然视频，或者说相机拍摄的视频而言，预测块与原始块通常存在差异。视频中不规则的运动、扭曲形变、遮挡、亮度等的变化，这些变化很难被完全预测。因此，混合编码框架会将当前块的原始块减去预测块得到残差块。残差块通常要比原始块简单很多，因而采用预测的方式可以显著提升压缩效率。在得到残差块之后，编码器通常不是对残差块直接进行编码，而是先进行变换。变换是将残差块从空间域变换到频率域。残差块变换到频率域以后，由于能量大多集中在低频区域，因此变换后的非零系数大多集中在左上角。接下来，可以利用量化来进一步压缩这些非零系数。此外，由于人眼对高频区域不敏感，因此，高频区域中的量化系数可以使用更大的量化步长。
下面结合图4A和图4B，对变换过程进行更为详细地举例说明。图4A为原始图像。图4B为DCT变换之后得到的图像。从图4B可以看出，原始图像经过DCT变换之后，只有左上角区域存在非零系数。当然，图4A和图4B采用的是对整幅图像进行DCT变换为例进行说明的，而在视频编解码中，图像通常是分割分块进行处理的，因而变换也是基于块来进行的。
通常来说，变换在视频的压缩中非常有用。但是，也并非所有的编码块都必须要做变换。在有些情况下，对编码块进行变换反而会导致压缩效果变差。因此，在某些标准(例如VVC)中，编码器可以选择是否对当前块进行变换操作。
接下来，介绍一下编解码过程中常用的变换方式。
DCT2是视频压缩标准中最常用的变换。其变换的基图像如图5所示。VVC中还可以使用DCT8和 离散正弦变换(discrete sine transform，DST)7。以上这些变换的基本公式如表一所示。
表一
图像一般是二维的，如果对图像直接进行二维变换，则变换过程的运算量和内存开销均非常大，目前的硬件条件暂时无法支持这种二维变换。因此，目前标准中的变换均采用一维变换。也就是说，将二维变换拆分成水平方向和竖直方向的一维变换，并分成两步进行。例如，可以先进行水平方向的变换再进行竖直方向的变换。或者，也可以先进行竖直方向的变换再进行水平方向的变换。
在VVC中，支持DCT2、DCT8和DST7等多种变换方式。在每一种变换方式中，可以包括多个变换核。变换核可以单独应用于水平方向或竖直方向的变换。对于一个编码块，编码器可以选择合适的变换核并将该变换核的索引传输到码流中，解码器根据索引即可确定反变换的变换核。水平方向和竖直方向的变换可以选择不同的变换核。例如，水平方向可以使用DCT8进行变换，竖直方向可以使用DST7进行变换。从上文提及的多种变换方式选择合适的变换方式的技术通常称为多变换选择(multiple transform selection，MTS)。
在VVC中，用语法元素例如mts_idx来确定基础变换的变换核。如表二所示，trTypeHor表示水平变换的变换核；trTypeVer表示竖直变换的变换核。在表二中，trTypeHor和trTypeVer的取值的含义为：0表示DCT2型变换；1表示DST7型变换；2表示DCT8型变换。
表二
基于DCT对水平方向和竖直方向的图像纹理进行变换的效果比较好。然而，基于DCT对斜向的图像纹理进行变换的效果会差一些。由于水平方向和竖直方向的图像纹理是最常见的，因而DCT对提升压缩效率是非常有用的。随着视频编解码技术对压缩效率需求的不断提高，如果能够对斜向的图像纹理进行更有效地处理，可以进一步提升压缩效率。
因此，VVC中引入了LFNST，从而更有效地处理斜向的图像纹理。下面结合图6A和图6B对LFNST进行详细的介绍。图6A示出的是不包括LFNST的普通编解码过程。如图6A所示，在编码过程中，只进行了一次基础变换(primary transform)。相应地，在解码过程中，也只进行了一次基础变换的反变换。图6B示出的是引入LFNST之后的编解码过程。如图6B所示，在编码过程中，在执行量化操作之前，先执行基于DCT2的变换，然后再执行基于LFNST的变换。在解码过程中，在执行反量化操作之后，先执行基于LFNST的反变换，然后再执行基于DCT2的反变换。基于以上介绍可知，LFNST是在基础变换(例如DCT2)的基础上再次进行的变换，所以LFNST通常被称为二次变换。上文提及的基础变换可以包括DCT2、DCT8和DST7等变换方式。
由上文的介绍可知，在编码端，LFNST用于对基础变换后的低频系数进行二次变换。基础变换通过对图像进行去相关性，把能量集中到图像的左上角。而LFNST对基础变换后的低频系数再次进行变换，以去除相关性。下面结合图7对LFNST在编解码过程中的输入和输出进行详细的介绍。
参见图7，在编码端，如果将16个系数输入到4×4的LFNST，则可以输出8个系数；如果将64个系数输入到8×8的LFNST，则可以输出16个系数。相应地，在解码端，如果将8个系数输入到4×4的反LFNST，则可以输出16个系数；如果将16个系数输入到8×8的反LFNST，则可以输出64个系数。
图8是LFNST在VVC中的一些基图像。从图8可以看出，变换核组1至3对应的基图像中有一 些明显的斜向纹理。也就是说，斜向的图像纹理基于变换核组1至3进行变换的效果更好。而平坦渐变的图像纹理基于变换核组0进行变换的效果更好。
LFNST所选用的变换核可以基于帧内预测模式确定。具体来说，角度预测按照指定的角度将参考像素平铺到当前块作为预测值，这意味着预测块会有明显的方向纹理。而当前块经过角度预测后的残差块在统计上也会表现出明显的角度特性。因此，LFNST所选用的变换核可以与帧内预测模式进行绑定。也就是说，在确定了帧内预测模式以后，LFNST可以使用帧内预测模式对应的一组(set)变换核。
VVC中的LFNST有4组变换核，每组包括2个变换核。表三给出了帧内预测模式和变换核组的对应关系。需要理解的是，在该表中，预测模式81至83为色度块进行帧内预测时使用的跨分量预测模式，而亮度块进行帧内预测时并不使用这几种预测模式。
可以对LFNST的变换核进行转置，从而可以使用一个变换核组处理更多的角度。举个例子，预测模式13至23和预测模式45至55均可以对应变换核组2，但是预测模式13至23明显是更接近于水平预测模式，而预测模式45至55明显是更接近于垂直预测模式。
表三
基于上文的介绍可知，在VVC中，基于帧内预测模式和LFNST的变换核之间的相关性，可确定LFNST使用哪一组变换核，从而减少了码流中传输的变换核个数。另外，可以通过码流和一些条件来确定编码块是否会使用LFNST；如果使用LFNST，还可以进一步确定编码块是使用一个变换核组中的第一个变换核，还是第二个变换核。
在相关技术中，LFNST的变换核组进行了进一步扩展，从而支持更多的变换核组。具体来说，在相关技术中的LFNST有35组变换核组(每个变换核组可以包括3个变换核)，变换核组和帧内预测模式的对应关系如表四所示。由于引入了更多的变换核组，因此每个变换核组对相应纹理的处理会更加高效。
表四
上文介绍了基础变换和二次变换。然而，这种基础变换(例如DCT2)加二次变换(例如LFNST)。的组合虽然具有很好的变换性能，但多次变换会引入更复杂的计算，因此基础变换加二次变换可以理解为一种性能和复杂度折衷的方案。相较于基础变换加二次变换，NSPT不但能够处理斜向纹理，而且不需要进行多次变换。下面对相关技术相关技术提供的NSTP进行介绍。
在相关技术中，NSPT与LFNST匹配变换核组的方式相同，也即是，根据帧内预测模式匹配变换核组。具体的匹配方法可以参照LFNST的方法，即每个变换核组包括3个变换核。相关技术中的一个NSPT的8×8基图像如图9所示。图9对应帧间角度预测模式7。从图9所示可以看出，NSPT的很多基图像具有明显的斜向纹理，因此NSTP对图像的斜向纹理的处理是很有效的。
在编码端，NSPT将空间域的残差变换到频率域的系数。基于NSPT进行变换的输入是全部可能的残差值，例如4×8的块有32个像素，那么编码端的NSPT就有32个输入。又例如，8×16的块有128 个像素，那么编码端的NSPT就有128个输入。编码端的输出可以比输入少，这样可以减少码流中写入的系数的数量上限，从而减少变换过程的计算量和变换核的存储空间。当然这样也会造成失真(distortion)，但是因为有量化的存在，这样做可以达到更好的性价比。
相应地，在解码端，NSPT将频率域的系数变换到空间域的残差。基于NSPT进行反变换的输出是残差值。表五是相关技术中的NSPT在解码端的变换核信息表。参见表五，4×8的块有32个像素，那么解码端的NSPT就有32个输出。8×16的块有128个像素，那么解码端NSPT就有128个输出。解码端的输入可以比输出少。
表五
由表五可知，NSPT对应的变换核组数和每组变换核数和LFNST是一样的。当然，这在以后的设计中有可能被改变，本申请实施例对此不作具体限定。解码端的输入个数是由具体的变换核设计决定的，这在以后的设计中也可能被改变，本申请实施例对此不作具体限定。例如，对4×8的块，如表五所示，现在的设计是20个输入，将来也可以设计成16个输入。输出个数是由块的尺寸决定的。具体来说，解码端的NSPT反变换过程是矩阵乘法过程。
变换核就是(输入个数×输出个数)的矩阵，因此(输入个数×输出个数)越大，变换核需要的存储空间也就越大，计算量也越大。考虑到计算复杂度，在相关技术中，NSPT仅应用于一些小尺寸的当前块，而大尺寸的当前块仍然使用DCT2+LFNST。这里提到的小尺寸例如包括4×4、4×8、8×4、8×8、4×16、16×4、8×16、16×8、4×32、32×4、8×32以及32×8。下面介绍NSPT在编解码过程中的运用。
在编码过程中，首先由编码器产生残差块。残差块可以由当前块的原始值减去预测值得到。
随后对残差块进行变换。如果当前块的尺寸符合NSPT的应用条件，则当前块可以使用NSPT；否则，当前块可以使用基础变换加二次变换(例如LFNST)的组合。编码器对残差块进行变换可以有多种选择。例如，可以只使用基础变换(例如DCT2、DST7和DCT8等)。或者，也可以根据当前块的尺寸使用NSPT或基础变换加二次变换(例如LFNST)的组合。举个例子，在相关技术中，4×4、4×8、8×4、8×8、4×16、16×4、8×16、16×8、4×32、32×4、8×32以及32×8的块尺寸可以使用NSPT，而其他块尺寸可以使用DCT2+LFNST。当然，当前块还可以跳过变换流程，也就是不进行变换。编码器可以选择合适的变换模式进行变换。
最后对变换得到的系数进行量化，熵编码等。
在解码过程中，首先由解码器通过熵解码，反量化得到系数。
随后对系数进行反变换得到残差块。解码器可以根据码流的指示和已知信息确定变换模式。例如，可以选择的变换模式包括DCT2、DST7、DCT8、DCT2+LFNST以及NSPT等。解码器可以根据块尺寸区分NSPT或基础变换加二次变换(例如LFNST)的组合。举个例子，在相关技术中，4×4、4×8、8×4、8×8、4×16、16×4、8×16、16×8、4×32、32×4、8×32以及32×8的块尺寸可以使用NSPT，其他块尺寸可以使用DCT2+LFNST。
最后将残差块和预测块相加得到重建块。
根据上文的介绍可知，当前块的尺寸越大，对应的残差块的尺寸也就越大。在编解码过程中，残差块的尺寸越大，进行变换和反变换时的计算复杂度也会相应地增加。正是由于该原因，很多变换性能较好的变换方式会仅应用于尺寸较小的当前块。以前文提到的NSPT为例，理论上，NSPT可以比基础变换(DCT2)+二次变换(LFNST)提供更高的压缩效率。但是，正是考虑到在大块上应用NSPT时的复杂度较高，相关技术仅在小块上使用NSPT，而在大块上仍使用DCT2+LFNST。
针对上述问题，本申请实施例提供一种编码方法，包括：确定当前块对应的第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；根据所述第二残差块确定第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；对所述第一残差块进行变换。
此外，本申请实施例还提供一种解码方法，包括：对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；根据所述第一残差块确定第二残差块，所述第 二残差块的尺寸等于所述当前块的尺寸；根据所述第二残差块确定所述当前块对应的重建块。
本申请实施例在编解码过程中，对尺寸较小的块进行变换或反变换，有助于降低变换或反变换过程的计算复杂度。
下文结合图10对本申请实施例的解码方法进行详细地举例说明。
图10是本申请实施例提供的解码方法的流程示意图。图10的方法可应用于解码器。
参见图10，在步骤S1010中，对变换系数进行反变换，确定当前块对应的第一残差块。例如，可以先解析码流，确定变换系数；然后，可以对变换系数进行反变换，确定第一残差块。
当前块可以指待解码的当前块。在一些实现方式中，当前块为亮度块。在另一些实现方式中，当前块也可以为色度块。
第一残差块的尺寸小于当前块的尺寸。在一些实现方式中，第一残差块的尺寸可以与当前块的尺寸具有关联关系。或者说，第一残差块的尺寸可以基于当前块的尺寸确定。例如，第一残差块的尺寸可以与当前块的尺寸存在比例关系。假设第一残差块的宽度和高度的比例为第一比例，当前块的宽度和高度的比例为第二比例，则第一比例可以等于第二比例。
作为示例，如果当前块的尺寸为16×16，则第一残差块的尺寸为8×8；如果当前块的尺寸为32×32，则第一残差块的尺寸为8×8；如果当前块的尺寸为64×64，则第一残差块的尺寸为8×8；如果当前块的尺寸为8×32，则第一残差块的尺寸为4×16；如果当前块的尺寸为32×8，则第一残差块的尺寸为16×4；如果当前块的尺寸为8×64，则第一残差块的尺寸为4×32；如果当前块的尺寸为64×8，则第一残差块的尺寸为32×4；如果当前块的尺寸为32×64，则第一残差块的尺寸为8×16；如果当前块的尺寸为64×32，则第一残差块的尺寸为16×8。
继续参见图10，在步骤S1020中，根据第一残差块确定第二残差块。第二残差块的尺寸等于当前块的尺寸。或者说，第二残差块的尺寸大于第一残差块的尺寸。例如，可以通过对第一残差块进行上采样的方式，确定第二残差块。对第一残差块进行上采样的方式可以有多种。例如，针对第一残差块，先沿水平方向进行上采样，再沿竖直方向进行上采样。又如，针对第一残差块，先沿竖直方向进行上采样，再沿水平方向进行上采样。沿着一维方向进行上采样，可以降低计算复杂度。当然，在一些实现方式中，也可以对第一残差块进行二维上采样，这样有助于提高采样精度。
上采样的方式可以有多种。例如，可以基于线性插值对第一残差块进行上采样。又如，可以基于插值滤波对第一残差块进行上采样。
在步骤S1030中，根据第二残差块确定当前块对应的重建块。例如，可以先对当前块进行预测，确定当前块对应的预测块。然后，可以根据第二残差块和预测块确定重建块。根据第二残差块和预测块确定重建块的方式例如可以包括：将第二残差块和预测块的像素之和确定为重建块。
本申请实施例提供的解码方法中，反变换操作之后得到的是尺寸较小的残差块(相对当前块而言尺寸较小)，也就是说，该反变换操作是针对尺寸较小的块所进行的反变换操作，这样有助于降低反变换过程的计算复杂度。
本申请实施例对步骤S1010所执行的反变换操作的类型不作具体限定。在一些实现方式中，可以将本申请实施例应用于反变换操作的复杂度与块的尺寸正相关关系明显的场景中。例如，前文提到，NSPT方案目前仅应用于尺寸较小的块，这是因为，块的尺寸越大，NSPT的计算复杂度也就越大，因此，可以将本申请实施例应用于基于NSPT的编解码场景中，从而降低变换/反变换操作的计算复杂度。
本申请实施例可适用于任意尺寸的当前块。在一些实现方式中，如果当前块的尺寸属于特定尺寸或特定尺寸集合，则使用图10所示的解码方法；如果当前块的尺寸不属于特定尺寸或特定尺寸集合，则可以直接通过反变换的方式确定与当前块尺寸相同的残差块。举个例子，如果当前块为小块，则可以通过对变换系数进行反变换获取与该小块的尺寸相同的残差块，并基于该残差块确定当前块对应的重建块；如果当前块为大块，则可以通过对变换系数进行反变换，确定比该大块的尺寸小的第一残差块。然后，基于第一残差块确定与大块尺寸相同的第二残差块，并根据第二残差块确定当前块对应的重建块。下面给出两种可能的实现方式。
实现方式一：
如果当前块的尺寸属于第一尺寸集合，则可以基于NSPT对变换系数进行反变换，确定第一残差块(尺寸小于当前块)。然后，可以根据第一残差块确定第二残差块，并根据第二残差块确定当前块对应的重建块。第二残差块和重建块的确定方式参见图10中的描述，此处不再重复描述。这里提到的第一尺寸集合例如可以是包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
如果当前块的尺寸不属于第一尺寸集合，则可以基于NSPT直接通过反变换的方式确定第三残差块(尺寸等于当前块)。然后，可以根据第三残差块确定当前块对应的重建块。例如，可以根据第三残差 块和预测块确定重建块。例如，如果当前块的尺寸为以下尺寸中的一种，则可以基于NSPT直接通过反变换的方式确定第三残差块：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
在上文中，当前块可以基于预设阈值判断是否属于第一尺寸集合。例如，如果当前块的尺寸大于或等于第一阈值，则当前块的尺寸属于第一尺寸集合。
根据上文的描述可以看出，在实现方式一中，无论当前块的尺寸是否属于第一尺寸集合，均基于NSPT进行反变换，这样可以充分利用NSPT压缩效率高的优势，整体上提升解码性能。
实现方式二：
如果当前块的尺寸属于第一尺寸集合，则可以基于NSPT对变换系数进行反变换，确定第一残差块(尺寸小于当前块)。然后，可以根据第一残差块确定第二残差块，并根据第二残差块确定当前块对应的重建块。第二残差块和重建块的确定方式参见图10中的描述，此处不再重复描述。这里提到的第一尺寸集合可以包括尺寸较大的块。例如，第一尺寸集合可以包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
如果当前块的尺寸属于第二尺寸集合，则可以基于NSPT直接通过反变换的方式确定第三残差块(尺寸等于当前块)。然后，可以根据第三残差块确定当前块对应的重建块。例如，可以根据第三残差块和预测块确定重建块。这里提到的第二尺寸集合可以包括尺寸相对较小的块。例如，第二尺寸集合可以是包括以下尺寸中的至少一种：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
如果当前块的尺寸属于第三尺寸集合，则可以基于基础变换加二次变换或者基础变换的方式进行反变换，确定第四残差块(尺寸等于当前块)。然后，可以根据第四残差块确定当前块对应的重建块。例如，可以根据第四残差块和预测块确定重建块。这里提到的第三尺寸集合例如可以包括除第一尺寸集合与第二尺寸集合内的尺寸之外的其他尺寸。例如，第三尺寸集合可以包括128×64。
在上文中，当前块可以基于预设阈值判断是否属于第一尺寸集合。例如，如果当前块的尺寸大于或等于第一阈值，则当前块的尺寸属于第一尺寸集合。
当然，当前块也可以基于预设阈值判断是否属于第二尺寸集合或者第三尺寸集合。例如，如果当前块的尺寸小于或等于第二阈值，则当前块的尺寸属于第二尺寸集合。
根据上文的描述可以看出，实现方式二提供了更多的反变换方式，从而可以根据实际情况灵活选择。
前文提到，可以通过对第一残差块进行上采样的方式，确定第二残差块。下文对上采样的方式进行更为具体的举例说明。
以上采样基于线性插值为例。如果对第一残差块进行一维的上采样，假设1：2的上采样，新插值出来的点的值可以用其相邻的两个点的值的平均值。假设1：4的上采样，新插值出来的点的值可以用其相邻的两个原始点的值的加权平均值，权重和距离成反比，即权重分别为3：1，2：2，1：3。
以对第一残差块进行水平方向的上采样为例，设上采样前的宽度为widthDwn，上采样后的宽度为widthUp，参数xUp＝widthUp/widthDwn。上采样前的残差数组为resDwn[widthDwn]，上采样后的残差数组为resUp[widthUp]。上采样参数之间的关系可以由如下的公式(1)、公式(2)和公式(3)表示。
其中，对x从0到widthDwn-1：
resUp[(x+1)×xUp-1]＝resDwn[x]       (1)
设resUp[-1]＝resDwn[0]，其中，resUp[-1]不算在残差块内，只是用来辅助计算，类似于填充(padding)。
对每一个m从0到widthDwn-1，记xHor＝m*xUp-1；对每一个dX从1到xUp-1。
sum＝(xUp-dX)×resUp[xHor]+dX×resUp[xHor+xUp]    (2)
resUp[xHor+dX]＝(sum+xUp/2)/xUp    (3)
以上采样基于插值滤波为例。例如，可以复用vvc中做分像素运动矢量运动补偿时使用的8抽头插值滤波。当然为了降低复杂度可以使用更小的抽头数，例如4抽头插值滤波等。
在一些实现方式中，图10的方法还可以包括：解析码流，确定量化系数；对量化系数进行反量化，确定变换系数。在得到变换系数之后，可以对变换系数进行反变换，确定当前块的第一残差块。
上文结合图10，详细描述了本申请实施例提供的解码方法。下面结合图11，详细描述本申请实施例提供的编码方法。
图11是本申请实施例提供的编码方法的流程示意图。图11的方法可应用于编码器。
参见图11，在步骤S1110中，确定当前块对应的第二残差块。第二残差块的尺寸等于当前块的尺寸。
当前块可以指待编码的当前块。在一些实现方式中，当前块为亮度块。在另一些实现方式中，当前块也可以为色度块。
在一些实现方式中，在执行步骤S1110之前，可以先对当前块进行预测，确定当前块对应的预测块；然后，根据预测块，确定第二残差块。根据预测块确定第二残差块的方式例如可以包括：将当前块和预测块之差确定为第二残差块。
在步骤S1120中，根据第二残差块确定第一残差块。第一残差块的尺寸小于当前块的尺寸。或者说，第二残差块的尺寸大于第一残差块的尺寸。例如，可以通过对第二残差块进行下采样的方式，确定第一残差块。对第二残差块进行下采样的方式可以有多种。例如，针对第二残差块，先沿水平方向进行下采样，再沿竖直方向进行下采样。又如，针对第二残差块，先沿竖直方向进行下采样，再沿水平方向进行下采样。沿着一维方向进行下采样，可以降低计算复杂度。当然，在一些实现方式中，也可以对第二残差块进行二维下采样，这样有助于提高采样精度。
下采样的方式可以有多种。例如，可以基于平均值对第二残差块进行下采样。又如，可以基于随机值对第二残差块进行下采样。
第一残差块的尺寸小于当前块的尺寸。在一些实现方式中，第一残差块的尺寸可以与当前块的尺寸具有关联关系。或者说，第一残差块的尺寸可以基于当前块的尺寸确定。假设第一残差块的宽度和高度的比例为第一比例，当前块的宽度和高度的比例为第二比例，则第一比例可以等于第二比例。
作为示例，如果当前块的尺寸为16×16，则第一残差块的尺寸为8×8；如果当前块的尺寸为32×32，则第一残差块的尺寸为8×8；如果当前块的尺寸为64×64，则第一残差块的尺寸为8×8；如果当前块的尺寸为8×32，则第一残差块的尺寸为4×16；如果当前块的尺寸为32×8，则第一残差块的尺寸为16×4；如果当前块的尺寸为8×64，则第一残差块的尺寸为4×32；如果当前块的尺寸为64×8，则第一残差块的尺寸为32×4；如果当前块的尺寸为32×64，则第一残差块的尺寸为8×16；如果当前块的尺寸为64×32，则第一残差块的尺寸为16×8。
继续参见图11，在步骤S1130中，对第一残差块进行变换。例如，可以对第一残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。
本申请实施例提供的编码方法中，在步骤S1120之后得到的是尺寸较小的残差块(相对当前块而言尺寸较小)。因此，在步骤S1130中的变换操作是针对尺寸较小的块所进行的变换操作，这样有助于降低变换过程的计算复杂度。
本申请实施例对步骤S1130所执行的变换操作的类型不作具体限定。在一些实现方式中，可以将本申请实施例应用于变换操作的复杂度与块的尺寸正相关关系明显的场景中。例如，前文提到，NSPT方案目前仅应用于尺寸较小的块，这是因为，块的尺寸越大，NSPT的计算复杂度也就越大，因此，可以将本申请实施例应用于基于NSPT的编解码场景中，从而降低变换/反变换操作的计算复杂度。
本申请实施例可适用于任意尺寸的当前块。在一些实现方式中，如果当前块的尺寸属于特定尺寸或特定尺寸集合，则使用图11所示的编码方法；如果当前块的尺寸不属于特定尺寸或特定尺寸集合，则可以直接对第二残差块进行变换，确定当前块的变换系数。举个例子，如果当前块为小块，则可以确定与该小块尺寸相同的残差块，并基于该残差块进行变换；如果当前块为大块，则可以确定与该大块尺寸相同的第二残差块。然后，基于第二残差块确定比该大块的尺寸小的第一残差块，并根据第一残差块进行变换。下面给出两种可能的实现方式。
实现方式一：
如果当前块的尺寸属于第一尺寸集合，则可以基于NSPT对第一残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。这里提到的第一尺寸集合例如可以是包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
如果当前块的尺寸不属于第一尺寸集合，则可以基于NSPT直接对第二残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。例如，如果当前块的尺寸为以下尺寸中的一种，则可以基于NSPT直接对第二残差块进行变换，确定当前块的变换系数：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
在上文中，当前块可以基于预设阈值判断是否属于第一尺寸集合。例如，如果当前块的尺寸大于或等于第一阈值，则当前块的尺寸属于第一尺寸集合。
根据上文的描述可以看出，在实现方式一中，无论当前块的尺寸是否属于第一尺寸集合，均基于NSPT进行变换，这样可以充分利用NSPT压缩效率高的优势，整体上提升编码性能。
实现方式二：
如果当前块的尺寸属于第一尺寸集合，则可以基于NSPT对第一残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。这里提到的第一尺寸集合例如可以是包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64， 64×32。
如果当前块的尺寸属于第二尺寸集合，则可以基于NSPT直接对第二残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。这里提到的第二尺寸集合可以包括尺寸相对较小的块。例如，第二尺寸集合可以是包括以下尺寸中的至少一种：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
如果当前块的尺寸属于第三尺寸集合，则可以基于基础变换加二次变换或者基础变换的方式直接对第二残差块进行变换，确定当前块的变换系数。然后，可以对变换系数进行量化，确定量化系数；对量化系数进行编码。这里提到的第三尺寸集合例如可以包括除第一尺寸集合与第二尺寸集合内的尺寸之外的其他尺寸。例如，第三尺寸集合可以包括128×64。
在上文中，当前块可以基于预设阈值判断是否属于第一尺寸集合。例如，如果当前块的尺寸大于或等于第一阈值，则当前块的尺寸属于第一尺寸集合。
当然，当前块也可以基于预设阈值判断是否属于第二尺寸集合或者第三尺寸集合。例如，如果当前块的尺寸小于或等于第二阈值，则当前块的尺寸属于第二尺寸集合。
根据上文的描述可以看出，实现方式二提供了更多的变换方式，从而可以根据实际情况灵活选择。
前文提到，可以通过对第二残差块进行下采样的方式，确定第一残差块。下文对下采样的方式进行更为具体的举例说明。
以基于平均值对第二残差块进行下采样为例。如果对第二残差块进行一维的下采样，假设2：1的下采样，可以用相邻两个点的平均值作为下采样后的一个点的值。假设4：1的下采样，可以用相邻四个点的平均值作为下采样后的一个点的值。
以水平方向的下采样为例，设下采样前的宽度为widthOrg，下采样后的宽度为widthDwn，参数xDwn＝widthOrg/widthDown。下采样前的残差值数组为resOrg[widthOrg]，下采样后的残差值数组为resDwn[widthDwn]，下采样参数之间的关系可以由如下的公式(4)表示。
其中，对x从0到widthDwn-1：
在一些实现方式中，图11的方法还可以包括：对基于第一残差块确定的变换系数进行量化，确定量化系数；对量化系数进行编码。
上文分别从解码端和编码端介绍了本申请实施例提供的解码方法和编码方法。下文结合编解码流程给出本申请的一种更为具体的实施例。
编码端流程
编码器产生残差块。该残差块可以由当前块的原始值减去预测值得到。该残差块的尺寸等于当前块的尺寸。
当前块的尺寸可以是CU的尺寸。如果当前在编码的是色度分量，当前块的尺寸可以是色度分量块的尺寸。如果一个CU被分成了多个TU，那么当前块的尺寸也可以是TU的尺寸。
对残差块进行变换。编码器对残差块进行变换可以有多种选择。例如，只使用基础变换的DCT2或DST7，DCT8等。又例如，也可以使用NSPT。当然，还可以跳过变换，也就是不进行变换。编码器可以选择合适的变换方式进行变换。
以编码器选择NSPT进行变换为例，4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4的块尺寸下可以直接使用NSPT对残差块进行变换。而对一些尺寸较大的当前块，例如32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32的块可以先对残差块进行下采样到适配的块尺寸，再对缩小的残差块进行NSPT变换。
如图12所示，编码端和解码端都涉及到一个块尺寸的转换，我们将应用变换的块尺寸或者应用本申请实施例的变换块的尺寸叫做适配的块尺寸。编码器在变换之前把残差块下采样到适配的块尺寸，用适配的块尺寸进行变换，即变换块(tb transform block)尺寸等于适配的块尺寸，并进行后续的量化和熵编码。
本申请实施例可以将当前块的尺寸与适配的块尺寸预设定对应关系。具体的对应关系例如可以如表六所示。
表六

解码端流程
解码器通过熵解码，反量化得到变换系数。
对变换系数进行反变换得到残差块。解码器可以根据码流的指示和已知信息确定变换方式。
以编码器选择NSPT进行反变换为例，4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4的块尺寸下可以直接使用NSPT对残差块进行反变换。而对一些尺寸较大的当前块，例如32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32的块符合适配NSPT的条件，则可以基于表六确定其适配的块尺寸。对变换系数进行基于NSPT的反变换，得到适配的块尺寸的残差块。再对适配的块尺寸的残差块进行上采样得到当前块的残差块。
将残差块和预测块相加得到重建块。
在上文介绍中，让更大尺寸的当前块也使用NSPT，但不是直接使用更大的变换核，而是将小的变换核用到大尺寸的当前块上。具体地，在编码端，对某一个尺寸的残差块，将它下采样到一个更小的尺寸的残差块，然后用NSPT进行变换。对应地，在解码端，对系数用NSPT进行反变换得到残差块，将残差块进行上采样得到一个更大尺寸的残差块。关键点是用空间域的上采样和下采样对NSPT进行适配。这样本来用于小尺寸的当前块的变换核也能用在大尺寸的当前块上。当然上采样和下采样是有损的，会丢失一定的高频信息，但是在进行有损编码时本身NSPT和量化也会丢失一定的高频信息，因而在这一步产生失真是可以接受的。
可以理解的是，解码端对变换系数的“反变换”，在标准文本中也可称为“变换”。本申请实施例中的“变换”和“反变换”对应的是两个相反的过程，如“变换”将空间域的数值转换到频率域的系数，那么“反变换”将频率域的系数转换到空间域的数值。标准如果只规定解码，那么标准文本中的“变换”是解码的部分，指的是本文中的“反变换”。解码端对变换系数的“反变换”，在标准文本中也可称为“变换”。
上文结合图1至图12，详细描述了本申请的方法实施例，下面结合图13至图16，详细描述本申请的装置实施例。应理解，方法实施例的描述与装置实施例的描述相互对应，因此，未详细描述的部分可以参见前面方法实施例。
图13为本申请一个实施例提供的解码器的结构示意图。如图13所示，所述解码器1300包括：第一确定单元1310、第二确定单元1320以及第三确定单元1330。
第一确定单元1310，配置为对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸。
第二确定单元1320，配置为根据所述第一残差块确定第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸。
第三确定单元1330，根据所述第二残差块确定所述当前块对应的重建块。
在一些实现方式中，所述第一确定单元1310配置为：基于第一变换方式对所述变换系数进行反变换，所述第一变换方式为不可分离的基础变换。
在一些实现方式中，所述解码器1300还包括：第四确定单元，配置为确定所述当前块的尺寸是否属于第一尺寸集合；所述第一确定单元1310配置为：如果所述当前块的尺寸属于所述第一尺寸集合，则确定所述第一残差块。
在一些实现方式中，所述解码器1300还包括：第五确定单元，配置为如果所述当前块的尺寸不属于所述第一尺寸集合，则对变换系数进行反变换，确定第三残差块，所述第三残差块的尺寸等于所述当前块的尺寸。
在一些实现方式中，所述第五确定单元配置为：基于第一变换方式对所述变换系数进行反变换，所述第一变换方式为不可分离的基础变换。
在一些实现方式中，所述解码器1300还包括：反变换单元，配置为：如果所述当前块的尺寸属于第二尺寸集合，则基于第一变换方式对变换系数进行反变换；和/或如果所述当前块的尺寸属于第三尺寸集合，则基于第二变换方式对变换系数进行反变换；其中，所述第一变换方式与所述第二变换方式不同。
在一些实现方式中，所述第一变换方式为不可分离的基础变换，所述第二变换方式为基础变换和二次变换的组合。
在一些实现方式中，所述第一尺寸集合包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
在一些实现方式中，所述第二尺寸集合包括以下尺寸中的至少一种：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
在一些实现方式中，所述第一残差块的尺寸基于所述当前块的尺寸确定。
在一些实现方式中，所述第一残差块的宽度和高度的比例为第一比例，所述当前块的宽度和高度的比例为第二比例，所述第一比例等于所述第二比例。
在一些实现方式中，如果所述当前块的尺寸为16×16，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为32×32，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为64×64，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为8×32，则所述第一残差块的尺寸为4×16；或者，如果所述当前块的尺寸为32×8，则所述第一残差块的尺寸为16×4；或者，如果所述当前块的尺寸为8×64，则所述第一残差块的尺寸为4×32；或者，如果所述当前块的尺寸为64×8，则所述第一残差块的尺寸为32×4；或者，如果所述当前块的尺寸为32×64，则所述第一残差块的尺寸为8×16；或者，如果所述当前块的尺寸为64×32，则所述第一残差块的尺寸为16×8。
在一些实现方式中，所述第二确定单元1320配置为：对所述第一残差块进行上采样，确定所述第二残差块。
在一些实现方式中，所述第二确定单元1320配置为：针对所述第一残差块，先沿水平方向进行上采样，再沿竖直方向进行上采样；或者，针对所述第一残差块，先沿竖直方向进行上采样，再沿水平方向进行上采样；或者，针对所述第一残差块，进行二维上采样。
在一些实现方式中，所述解码器1300还包括：第六确定单元，配置为：解析码流，确定量化系数；第七确定单元，配置为：对所述量化系数进行反量化，确定所述变换系数。
在一些实现方式中，所述第三确定单元1330配置为：对所述当前块进行预测，确定所述当前块对应的预测块；根据所述第二残差块和所述预测块确定所述重建块。
在一些实现方式中，所述第一确定单元1310配置为：所述第一残差块基于不可分离的基础变换确定，所述不可分离的基础变换的变换核组基于所述预测块的帧内预测模式确定。
可以理解地，在本申请实施例中，“单元”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是模块，还可以是非模块化的。而且在本实施例中的各组成部分可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
因此，本申请实施例提供了一种计算机可读存储介质，应用于解码器1200，该计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现前述第一实施例中的解码方法。
基于上述解码器1300的组成以及计算机可读存储介质，参见图14，其示出了本申请实施例提供的解码器1400的具体硬件结构示意图。如图14所示，解码器1400可以包括：通信接口1410、存储器1420和处理器1430；各个组件通过总线系统1440耦合在一起。可理解，总线系统1440用于实现这些组件之间的连接通信。总线系统1440除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图14中将各种总线都标为总线系统1440。其中，
通信接口1410，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送。
存储器1420，用于存储计算机程序。
处理器1430，用于在运行所述计算机程序时，执行：对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；根据所述第一残差块确定第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；根据所述第二残差块确定所述当前块对应的重建块。
可以理解，本申请实施例中的存储器1420可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可 编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请描述的系统和方法的存储器1420旨在包括但不限于这些和任意其它适合类型的存储器。
而处理器1430可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1430中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1430可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1420，处理器1430读取存储器1420中的信息，结合其硬件完成上述方法的步骤。
可以理解的是，本申请描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processing，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、微处理器、用于执行本申请所述功能的其它电子单元或其组合中。对于软件实现，可通过执行本申请所述功能的模块(例如过程、函数等)来实现本申请所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
可选地，作为另一个实施例，处理器1430还配置为在运行所述计算机程序时，执行前述实施例所述的解码方法。
图14为本申请一个实施例提供的编码器的结构示意图。如图15所示，所述编码器1500包括：第一确定单元1510、第二确定单元1520以及变换单元1530。
第一确定单元1510，配置为确定当前块对应的第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸。
第二确定单元1520，配置为根据所述第二残差块确定第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸。
变换单元1530，配置为对所述第一残差块进行变换。
在一些实现方式中，所述变换单元1530配置为：基于第一变换方式对所述第一残差块进行变换，所述第一变换方式为不可分离的基础变换。
在一些实现方式中，所述编码器1500还包括：第三确定单元，配置为：确定所述当前块的尺寸是否属于第一尺寸集合；所述第二确定单元1520配置为：如果所述当前块的尺寸属于所述第一尺寸集合，则根据所述第二残差块确定所述第一残差块。
在一些实现方式中，所述编码器1500还包括：第一变换单元，配置为：如果所述当前块的尺寸不属于所述第一尺寸集合，则对所述第二残差块进行变换。
在一些实现方式中，所述第一变换单元配置为：基于第一变换方式对所述第二残差块进行变换，所述第一变换方式为不可分离的基础变换。
在一些实现方式中，所述编码器1500还包括：第二变换单元，配置为：如果所述当前块的尺寸属于第二尺寸集合，则基于第一变换方式对所述第二残差块进行变换；和/或如果所述当前块的尺寸属于第三尺寸集合，则基于第二变换方式对所述第二残差块进行变换；其中，所述第一变换方式与所述第二变换方式不同。
在一些实现方式中，所述第一变换方式为不可分离的基础变换，所述第二变换方式为基础变换和二次变换的组合。
在一些实现方式中，所述第一尺寸集合包括以下尺寸中的至少一种：32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
在一些实现方式中，所述第二尺寸集合包括以下尺寸中的至少一种：4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
在一些实现方式中，所述第一残差块的尺寸基于所述当前块的尺寸确定。
在一些实现方式中，所述第一残差块的宽度和高度的比例为第一比例，所述当前块的宽度和高度的比例为第二比例，所述第一比例等于所述第二比例。
在一些实现方式中，如果所述当前块的尺寸为16×16，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为32×32，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为64×64，则所述第一残差块的尺寸为8×8；或者，如果所述当前块的尺寸为8×32，则所述第一残差块的尺寸为4×16；或者，如果所述当前块的尺寸为32×8，则所述第一残差块的尺寸为16×4；或者，如果所述当前块的尺寸为8×64，则所述第一残差块的尺寸为4×32；或者，如果所述当前块的尺寸为64×8，则所述第一残差块的尺寸为32×4；或者，如果所述当前块的尺寸为32×64，则所述第一残差块的尺寸为8×16；或者，如果所述当前块的尺寸为64×32，则所述第一残差块的尺寸为16×8。
在一些实现方式中，所述第二确定单元1520配置为：对所述第二残差块进行下采样，确定所述第一残差块。
在一些实现方式中，所述第二确定单元1520配置为：针对所述第一残差块，先沿水平方向进行下采样，再沿竖直方向进行下采样；或者，针对所述第一残差块，先沿竖直方向进行下采样，再沿水平方向进行下采样；或者，针对所述第一残差块，进行二维下采样。
在一些实现方式中，所述编码器1500还包括：第四确定单元，配置为：对基于所述第一残差块确定的变换系数进行量化，确定量化系数；编码单元，配置为：对所述量化系数进行编码。
在一些实现方式中，所述第一确定单元1510配置为：对所述当前块进行预测，确定所述当前块对应的预测块；根据所述预测块，确定所述第二残差块。
在一些实现方式中，所述变换单元1530配置为：基于不可分离的基础变换对所述第一残差块进行变换，所述不可分离的基础变换的变换核组基于所述预测块的帧内预测模式确定。
可以理解地，在本申请实施例中，“单元”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是模块，还可以是非模块化的。而且在本实施例中的各组成部分可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
因此，本申请实施例提供了一种计算机可读存储介质，应用于编码器1500，该计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现前述第一实施例中的解码方法。
基于上述编码器1500的组成以及计算机可读存储介质，参见图16，其示出了本申请实施例提供的编码器1600的具体硬件结构示意图。如图16所示，编码器1600可以包括：通信接口1610、存储器1620和处理器1630；各个组件通过总线系统1640耦合在一起。可理解，总线系统1640用于实现这些组件之间的连接通信。总线系统1640除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图16中将各种总线都标为总线系统1640。其中，
通信接口1610，用于在与其他外部网元之间进行收发信息过程中，信号的接收和发送。
存储器1620，用于存储计算机程序。
处理器1630，用于在运行所述计算机程序时，执行：确定当前块对应的第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；根据所述第二残差块确定第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；对所述第一残差块进行变换。
可以理解，本申请实施例中的存储器1620可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic  RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请描述的系统和方法的存储器1620旨在包括但不限于这些和任意其它适合类型的存储器。
而处理器1630可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1630中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1630可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1620，处理器1630读取存储器1620中的信息，结合其硬件完成上述方法的步骤。
可以理解的是，本申请描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processing，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、微处理器、用于执行本申请所述功能的其它电子单元或其组合中。对于软件实现，可通过执行本申请所述功能的模块(例如过程、函数等)来实现本申请所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
可选地，作为另一个实施例，处理器1630还配置为在运行所述计算机程序时，执行前述实施例所述的编码方法。
需要说明的是，在本申请中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。
本申请所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。
本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (44)

  1. 一种解码方法，应用于解码器，包括：
    对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；
    根据所述第一残差块确定第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；
    根据所述第二残差块确定所述当前块对应的重建块。
  2. 根据权利要求1所述的方法，其中，所述对所述变换系数进行反变换，包括：
    基于第一变换方式对所述变换系数进行反变换，所述第一变换方式为不可分离的基础变换。
  3. 根据权利要求1或2所述的方法，其中，所述方法还包括：
    确定所述当前块的尺寸是否属于第一尺寸集合；
    所述确定当前块对应的第一残差块，包括：
    如果所述当前块的尺寸属于所述第一尺寸集合，则确定所述第一残差块。
  4. 根据权利要求3所述的方法，其中，所述方法还包括：
    如果所述当前块的尺寸不属于所述第一尺寸集合，则对变换系数进行反变换，确定第三残差块，所述第三残差块的尺寸等于所述当前块的尺寸。
  5. 根据权利要求4所述的方法，其中，所述对变换系数进行反变换，确定第三残差块，包括：
    基于第一变换方式对所述变换系数进行反变换，确定所述第三残差块，所述第一变换方式为不可分离的基础变换。
  6. 根据权利要求3所述的方法，其中，所述方法还包括：
    如果所述当前块的尺寸属于第二尺寸集合，则基于第一变换方式对变换系数进行反变换；和/或
    如果所述当前块的尺寸属于第三尺寸集合，则基于第二变换方式对变换系数进行反变换；
    其中，所述第一变换方式与所述第二变换方式不同。
  7. 根据权利要求6所述的方法，其中，所述第一变换方式为不可分离的基础变换，所述第二变换方式为基础变换和二次变换的组合或者基础变换。
  8. 根据权利要求3至7任一项所述的方法，其中，所述确定所述当前块的尺寸是否属于第一尺寸集合，包括：
    如果所述当前块的尺寸大于或等于第一阈值，则所述当前块的尺寸属于所述第一尺寸集合。
  9. 根据权利要求8所述的方法，其中，所述第一尺寸集合包括以下尺寸中的至少一种：
    32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
  10. 根据权利要求6或7所述的方法，其中，所述如果所述当前块的尺寸属于第二尺寸集合，包括：
    如果所述当前块的尺寸小于或等于第二阈值，则所述当前块的尺寸属于所述第二尺寸集合。
  11. 根据权利要求10所述的方法，其中，所述第二尺寸集合包括以下尺寸中的至少一种：
    4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
  12. 根据权利要求1至11任一项所述的方法，其中，所述第一残差块的尺寸基于所述当前块的尺寸确定。
  13. 根据权利要求12所述的方法，其中，所述第一残差块的宽度和高度的比例为第一比例，所述当前块的宽度和高度的比例为第二比例，所述第一比例等于所述第二比例。
  14. 根据权利要求1至13任一项所述的方法，其中：
    如果所述当前块的尺寸为16×16，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为32×32，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为64×64，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为8×32，则所述第一残差块的尺寸为4×16；或者，
    如果所述当前块的尺寸为32×8，则所述第一残差块的尺寸为16×4；或者，
    如果所述当前块的尺寸为8×64，则所述第一残差块的尺寸为4×32；或者，
    如果所述当前块的尺寸为64×8，则所述第一残差块的尺寸为32×4；或者，
    如果所述当前块的尺寸为32×64，则所述第一残差块的尺寸为8×16；或者，
    如果所述当前块的尺寸为64×32，则所述第一残差块的尺寸为16×8。
  15. 根据权利要求1至14任一项所述的方法，其中，所述根据所述第一残差块确定第二残差块，包括：
    对所述第一残差块进行上采样，确定所述第二残差块。
  16. 根据权利要求15所述的方法，其中，所述对所述第一残差块进行上采样，包括：
    针对所述第一残差块，先沿水平方向进行上采样，再沿竖直方向进行上采样；或者，
    针对所述第一残差块，先沿竖直方向进行上采样，再沿水平方向进行上采样；或者，
    针对所述第一残差块，进行二维上采样。
  17. 根据权利要求1至16中任一项所述的方法，其中，所述方法还包括：
    解析码流，确定量化系数；
    对所述量化系数进行反量化，确定所述变换系数。
  18. 根据权利要求1至17任一项所述的方法，其中，所述根据所述第二残差块确定所述当前块对应的重建块，包括：
    对所述当前块进行预测，确定所述当前块对应的预测块；
    根据所述第二残差块和所述预测块确定所述重建块。
  19. 根据权利要求1至18任一项所述的方法，其中，所述第一残差块基于不可分离的基础变换确定，所述不可分离的基础变换的变换核组基于所述预测块的帧内预测模式确定。
  20. 一种编码方法，应用于编码器，包括：
    确定当前块对应的第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；
    根据所述第二残差块确定第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；
    对所述第一残差块进行变换。
  21. 根据权利要求20所述的方法，其中，所述对所述第一残差块进行变换，包括：
    基于第一变换方式对所述第一残差块进行变换，所述第一变换方式为不可分离的基础变换。
  22. 根据权利要求20或21所述的方法，其中，所述方法还包括：
    确定所述当前块的尺寸是否属于第一尺寸集合；
    所述根据所述第二残差块确定第一残差块，包括：
    如果所述当前块的尺寸属于所述第一尺寸集合，则根据所述第二残差块确定所述第一残差块。
  23. 根据权利要求22所述的方法，其中，所述方法还包括：
    如果所述当前块的尺寸不属于所述第一尺寸集合，则对所述第二残差块进行变换。
  24. 根据权利要求23所述的方法，其中，所述对所述第二残差块进行变换，包括：
    基于第一变换方式对所述第二残差块进行变换，所述第一变换方式为不可分离的基础变换。
  25. 根据权利要求22所述的方法，其中，所述方法还包括：
    如果所述当前块的尺寸属于第二尺寸集合，则基于第一变换方式对所述第二残差块进行变换；和/或
    如果所述当前块的尺寸属于第三尺寸集合，则基于第二变换方式对所述第二残差块进行变换；
    其中，所述第一变换方式与所述第二变换方式不同。
  26. 根据权利要求25所述的方法，其中，所述第一变换方式为不可分离的基础变换，所述第二变换方式为基础变换和二次变换的组合或者基础变换。
  27. 根据权利要求22至26任一项所述的方法，其中，所述确定所述当前块的尺寸是否属于第一尺寸集合，包括：
    如果所述当前块的尺寸大于或等于第一阈值，则所述当前块的尺寸属于所述第一尺寸集合。
  28. 根据权利要求27所述的方法，其中，所述第一尺寸集合包括以下尺寸中的至少一种：
    32×32，64×64，8×32，32×8，8×64，64×8，32×64，64×32。
  29. 根据权利要求25或26所述的方法，其中，所述如果所述当前块的尺寸属于第二尺寸集合，包括：
    如果所述当前块的尺寸小于或等于第二阈值，则所述当前块的尺寸属于所述第二尺寸集合。
  30. 根据权利要求29所述的方法，其中，所述第二尺寸集合包括以下尺寸中的至少一种：
    4×4，4×8，8×4，8×8，4×16，16×4，8×16，16×8，4×32，32×4。
  31. 根据权利要求20至30任一项所述的方法，其中，所述第一残差块的尺寸基于所述当前块的尺寸确定。
  32. 根据权利要求31所述的方法，其中，所述第一残差块的宽度和高度的比例为第一比例，所述当前块的宽度和高度的比例为第二比例，所述第一比例等于所述第二比例。
  33. 根据权利要求20至32任一项所述的方法，其中：
    如果所述当前块的尺寸为16×16，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为32×32，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为64×64，则所述第一残差块的尺寸为8×8；或者，
    如果所述当前块的尺寸为8×32，则所述第一残差块的尺寸为4×16；或者，
    如果所述当前块的尺寸为32×8，则所述第一残差块的尺寸为16×4；或者，
    如果所述当前块的尺寸为8×64，则所述第一残差块的尺寸为4×32；或者，
    如果所述当前块的尺寸为64×8，则所述第一残差块的尺寸为32×4；或者，
    如果所述当前块的尺寸为32×64，则所述第一残差块的尺寸为8×16；或者，
    如果所述当前块的尺寸为64×32，则所述第一残差块的尺寸为16×8。
  34. 根据权利要求20至33任一项所述的方法，其中，所述根据所述第二残差块确定第一残差块，包括：
    对所述第二残差块进行下采样，确定所述第一残差块。
  35. 根据权利要求34所述的方法，其中，所述对所述第二残差块进行下采样，包括：
    针对所述第一残差块，先沿水平方向进行下采样，再沿竖直方向进行下采样；或者，
    针对所述第一残差块，先沿竖直方向进行下采样，再沿水平方向进行下采样；或者，
    针对所述第一残差块，进行二维下采样。
  36. 根据权利要求20至35任一项所述的方法，其中，所述方法还包括：
    对基于所述第一残差块确定的变换系数进行量化，确定量化系数；
    对所述量化系数进行编码。
  37. 根据权利要求20至36任一项所述的方法，其中，所述确定当前块对应的第二残差块，包括：
    对所述当前块进行预测，确定所述当前块对应的预测块；
    根据所述预测块，确定所述第二残差块。
  38. 根据权利要求20至37任一项所述的方法，其中，所述对所述第一残差块进行变换，包括：
    基于不可分离的基础变换对所述第一残差块进行变换，所述不可分离的基础变换的变换核组基于所述预测块的帧内预测模式确定。
  39. 一种解码器，包括：
    第一确定单元，配置为对变换系数进行反变换，确定当前块对应的第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；
    第二确定单元，配置为根据所述第一残差块确定第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；
    第三确定单元，配置为根据所述第二残差块确定所述当前块对应的重建块。
  40. 一种解码器，包括：
    存储器，用于存储计算机程序；
    处理器，用于在运行所述计算机程序时，执行如权利要求1至19中任一项所述的方法。
  41. 一种编码器，包括：
    第一确定单元，配置为确定当前块对应的第二残差块，所述第二残差块的尺寸等于所述当前块的尺寸；
    第二确定单元，配置为根据所述第二残差块确定第一残差块，所述第一残差块的尺寸小于所述当前块的尺寸；
    变换单元，配置为对所述第一残差块进行变换。
  42. 一种编码器，包括：
    存储器，用于存储计算机程序；
    处理器，用于在运行所述计算机程序时，执行如权利要求20至38中任一项所述的方法。
  43. 一种存储比特流的非易失性计算机可读存储介质，所述比特流通过利用编码器的编码方法而生成，或者，所述比特流通过利用解码器的解码方法而解码，其中，所述解码方法为如权利要求1至19中任一项所述的方法、所述编码方法为如权利要求20至38中任一项所述的方法。
  44. 一种码流，所述码流包括如权利要求1至19中任一项所述的方法生成的码流或者如权利要求20至38中任一项所述的方法生成的码流。
PCT/CN2023/135808 2023-12-01 2023-12-01 编解码方法、编解码器以及存储介质 Pending WO2025112031A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2023/135808 WO2025112031A1 (zh) 2023-12-01 2023-12-01 编解码方法、编解码器以及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2023/135808 WO2025112031A1 (zh) 2023-12-01 2023-12-01 编解码方法、编解码器以及存储介质

Publications (1)

Publication Number Publication Date
WO2025112031A1 true WO2025112031A1 (zh) 2025-06-05

Family

ID=95896129

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/135808 Pending WO2025112031A1 (zh) 2023-12-01 2023-12-01 编解码方法、编解码器以及存储介质

Country Status (1)

Country Link
WO (1) WO2025112031A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100009718A (ko) * 2008-07-21 2010-01-29 에스케이 텔레콤주식회사 예측 방향을 이용한 영상 부호화/복호화 장치 및 방법
US20110134999A1 (en) * 2009-12-09 2011-06-09 Samsung Electronics Co., Ltd. Method and apparatus for encoding video, and method and apparatus for decoding video
KR20140079882A (ko) * 2012-12-18 2014-06-30 한양대학교 산학협력단 적응적인 인트라 예측을 이용한 영상 부호화/복호화 장치 및 방법
CN112055210A (zh) * 2020-08-19 2020-12-08 浙江大华技术股份有限公司 一种视频图像处理方法、编码器和计算机可读存储介质
CN113966617A (zh) * 2019-06-14 2022-01-21 瑞典爱立信有限公司 对mip缩减预测的样本值裁剪
CN114830664A (zh) * 2019-12-30 2022-07-29 Oppo广东移动通信有限公司 变换方法、编码器、解码器以及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100009718A (ko) * 2008-07-21 2010-01-29 에스케이 텔레콤주식회사 예측 방향을 이용한 영상 부호화/복호화 장치 및 방법
US20110134999A1 (en) * 2009-12-09 2011-06-09 Samsung Electronics Co., Ltd. Method and apparatus for encoding video, and method and apparatus for decoding video
KR20140079882A (ko) * 2012-12-18 2014-06-30 한양대학교 산학협력단 적응적인 인트라 예측을 이용한 영상 부호화/복호화 장치 및 방법
CN113966617A (zh) * 2019-06-14 2022-01-21 瑞典爱立信有限公司 对mip缩减预测的样本值裁剪
CN114830664A (zh) * 2019-12-30 2022-07-29 Oppo广东移动通信有限公司 变换方法、编码器、解码器以及存储介质
CN112055210A (zh) * 2020-08-19 2020-12-08 浙江大华技术股份有限公司 一种视频图像处理方法、编码器和计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI MU, LI JINXING, GU SHUHANG, WU FENG, ZHANG DAVID: "End-to-End Optimized 360° Image Compression", IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE, USA, vol. 31, 1 January 2022 (2022-01-01), USA, pages 6267 - 6281, XP093319588, ISSN: 1057-7149, DOI: 10.1109/TIP.2022.3208429 *

Similar Documents

Publication Publication Date Title
US12192454B2 (en) Sample adaptive offset control
US20230353734A1 (en) Method and apparatus for encoding/decoding image information
CN113728636B (zh) 编解码视频中二次变换的选择性使用
US11778235B2 (en) Signaling coding of transform-skipped blocks
US20140376619A1 (en) Sample adaptive offset control
CN119653081A (zh) 编码方法、解码方法、编码器、解码器以及存储介质
TW202041006A (zh) 用於視頻編解碼的量化矩陣計算和表示的方法和裝置
CN114747216A (zh) 多重转换选择的发信技术
JP2025520767A (ja) デコーディング方法、エンコーディング方法、デコーダー及びエンコーダー
WO2023193253A1 (zh) 解码方法、编码方法、解码器以及编码器
US12389003B2 (en) Transform method, encoder, decoder, and storage medium
US12058373B2 (en) Transform method, encoder, decoder, and storage medium
EP4507295A1 (en) Decoding method, encoding method, decoder, and encoder
CN114175653B (zh) 用于视频编解码中的无损编解码模式的方法和装置
WO2025112031A1 (zh) 编解码方法、编解码器以及存储介质
WO2025129660A1 (zh) 编解码方法、编解码器、码流以及存储介质
WO2025091378A1 (zh) 编解码方法、编解码器以及存储介质
WO2025213368A1 (zh) 编解码方法、码流、编码器、解码器以及存储介质
WO2025145288A9 (zh) 编解码方法、码流、编码器、解码器以及存储介质
WO2025059901A1 (zh) 编解码方法、编解码器以及存储介质
WO2025073085A1 (zh) 编解码方法、编解码器以及存储介质
WO2025138170A1 (zh) 编解码方法、编解码器以及存储介质
WO2025148090A1 (zh) 编解码方法、编解码器、码流以及存储介质
WO2025199802A1 (zh) 编解码方法、码流、编码器、解码器以及存储介质
WO2024192733A9 (zh) 视频编解码方法、装置、设备、系统、及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23959939

Country of ref document: EP

Kind code of ref document: A1