WO2023173255A1 - Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage - Google Patents

Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage Download PDF

Info

Publication number
WO2023173255A1
WO2023173255A1 PCT/CN2022/080706 CN2022080706W WO2023173255A1 WO 2023173255 A1 WO2023173255 A1 WO 2023173255A1 CN 2022080706 W CN2022080706 W CN 2022080706W WO 2023173255 A1 WO2023173255 A1 WO 2023173255A1
Authority
WO
WIPO (PCT)
Prior art keywords
transformation
image area
current
flag
image
Prior art date
Application number
PCT/CN2022/080706
Other languages
English (en)
Chinese (zh)
Inventor
唐桐
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to PCT/CN2022/080706 priority Critical patent/WO2023173255A1/fr
Publication of WO2023173255A1 publication Critical patent/WO2023173255A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties

Definitions

  • the present application relates to the field of video coding and decoding technology, and in particular, to an image coding and decoding method, device, equipment, system, and storage medium.
  • Digital video technology can be incorporated into a variety of video devices, such as digital televisions, smartphones, computers, e-readers, or video players.
  • video data includes a larger amount of data.
  • video devices implement video compression technology to make the video data more efficiently transmitted or stored.
  • Video compression is achieved through coding, and the coding process includes prediction, transformation, and quantization. For example, determine the prediction block of the current block through intra-frame prediction and/or inter-frame prediction, subtract the prediction block from the current block to obtain the residual block, transform the residual block to obtain the transform coefficient, and quantize the transform coefficient to obtain the quantized coefficient, And the quantized coefficients are encoded to form a code stream.
  • MTS Multiple Transform Selection
  • Embodiments of the present application provide an image encoding and decoding method, device, equipment, system, and storage medium, which determine the transformation method of the current encoding block through the continuity parameters of the image area, avoid unnecessary calculation overhead, and improve the encoding speed.
  • the current coding block is transformed to obtain the transformation coefficient of the current coding block.
  • embodiments of the present application provide an image decoding method, including:
  • the first flag is determined based on the continuity parameter of the image area, and the first flag is used to indicate whether the image area is allowed to transform from M candidates
  • the transformation mode is determined in the mode, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;
  • the transformation coefficient of the current decoding block is inversely transformed to obtain the residual value of the current decoding block.
  • the present application provides an image encoding device for performing the method in the above first aspect or its respective implementations.
  • the encoder includes a functional unit for performing the method in the above-mentioned first aspect or its respective implementations.
  • the present application provides an image decoding device for performing the method in the above second aspect or its respective implementations.
  • the decoder includes a functional unit for performing the method in the above-mentioned second aspect or its respective implementations.
  • a video encoder including a processor and a memory.
  • the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the above first aspect or its respective implementations.
  • a sixth aspect provides a video decoder, including a processor and a memory.
  • the memory is used to store a computer program
  • the processor is used to call and run the computer program stored in the memory to execute the method in the above second aspect or its respective implementations.
  • a seventh aspect provides a video encoding and decoding system, including a video encoder and a video decoder.
  • the video encoder is used to perform the method in the above-mentioned first aspect or its various implementations
  • the video decoder is used to perform the method in the above-mentioned second aspect or its various implementations.
  • An eighth aspect provides a chip for implementing any one of the above-mentioned first to second aspects or the method in each implementation manner thereof.
  • the chip includes: a processor, configured to call and run a computer program from a memory, so that the device installed with the chip executes any one of the above-mentioned first to second aspects or implementations thereof. method.
  • a ninth aspect provides a computer-readable storage medium for storing a computer program that causes a computer to execute any one of the above-mentioned first to second aspects or the method in each implementation thereof.
  • a computer program product including computer program instructions, which enable a computer to execute any one of the above-mentioned first to second aspects or the methods in each implementation thereof.
  • An eleventh aspect provides a computer program that, when run on a computer, causes the computer to execute any one of the above-mentioned first to second aspects or the method in each implementation thereof.
  • a twelfth aspect provides a code stream, which is generated by any one of the above-mentioned first aspects or implementations thereof.
  • the continuity parameter is used to indicate the continuity of changes in the pixels in the image area; according to the continuity parameter of the image area, the transformation method of the current coding block is determined ; According to the transformation method, transform the current coding block to obtain the transformation coefficient of the current coding block. That is, in this embodiment of the present application, the transformation method of the current coding block is determined according to the size of the continuity parameter.
  • the default first transformation method can be used as The transformation method of the current coding block, and skips the operation of selecting the optimal transformation method from multiple transformation methods to save unnecessary computing overhead and improve coding efficiency.
  • Figure 1 is a schematic block diagram of a video encoding and decoding system related to an embodiment of the present application
  • Figure 2 is a schematic block diagram of a video encoder involved in an embodiment of the present application
  • Figure 3 is a schematic block diagram of a video decoder involved in an embodiment of the present application.
  • Figure 4 is a schematic flow chart of an image encoding method provided by an embodiment of the present application.
  • Figure 5 is a schematic flow chart of an image encoding method provided by an embodiment of the present application.
  • Figure 6 is a schematic flow chart of a video decoding method provided by an embodiment of the present application.
  • Figure 7 is a schematic flow chart of an image decoding method provided by an embodiment of the present application.
  • Figure 8 is a schematic block diagram of an image encoding device provided by an embodiment of the present application.
  • Figure 9 is a schematic block diagram of an image decoding device provided by an embodiment of the present application.
  • Figure 10 is a schematic block diagram of an electronic device provided by an embodiment of the present application.
  • Figure 11 is a schematic block diagram of a video encoding and decoding system provided by an embodiment of the present application.
  • This application can be applied to the fields of image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, real-time video encoding and decoding, etc.
  • the solution of this application can be combined with the audio and video coding standard (AVS for short), such as H.264/audio video coding (AVC for short) standard, H.265/high-efficiency video coding (AVS for short) high efficiency video coding (HEVC) standard and H.266/versatile video coding (VVC) standard.
  • AVC audio video coding
  • HEVC high efficiency video coding
  • VVC variatile video coding
  • the solution of this application can be operated in conjunction with other proprietary or industry standards, including ITU-TH.261, ISO/IECMPEG-1Visual, ITU-TH.262 or ISO/IECMPEG-2Visual, ITU-TH.263 , ISO/IECMPEG-4Visual, ITU-TH.264 (also known as ISO/IECMPEG-4AVC), including scalable video codec (SVC) and multi-view video codec (MVC) extensions.
  • SVC scalable video codec
  • MVC multi-view video codec
  • FIG. 1 For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced with reference to FIG. 1 .
  • Figure 1 is a schematic block diagram of a video encoding and decoding system related to an embodiment of the present application. It should be noted that Figure 1 is only an example, and the video encoding and decoding system in the embodiment of the present application includes but is not limited to what is shown in Figure 1 .
  • the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120 .
  • the encoding device is used to encode the video data (which can be understood as compression) to generate a code stream, and transmit the code stream to the decoding device.
  • the decoding device decodes the code stream generated by the encoding device to obtain decoded video data.
  • the encoding device 110 in the embodiment of the present application can be understood as a device with a video encoding function
  • the decoding device 120 can be understood as a device with a video decoding function. That is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120. Examples include smartphones, desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, vehicle-mounted computers, and the like.
  • the encoding device 110 may transmit the encoded video data (eg, code stream) to the decoding device 120 via the channel 130 .
  • Channel 130 may include one or more media and/or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120 .
  • channel 130 includes one or more communication media that enables encoding device 110 to transmit encoded video data directly to decoding device 120 in real time.
  • encoding device 110 may modulate the encoded video data according to the communication standard and transmit the modulated video data to decoding device 120.
  • the communication media includes wireless communication media, such as radio frequency spectrum.
  • the communication media may also include wired communication media, such as one or more physical transmission lines.
  • channel 130 includes a storage medium that can store video data encoded by encoding device 110 .
  • Storage media include a variety of local access data storage media, such as optical disks, DVDs, flash memories, etc.
  • the decoding device 120 may obtain the encoded video data from the storage medium.
  • channel 130 may include a storage server that may store video data encoded by encoding device 110 .
  • the decoding device 120 may download the stored encoded video data from the storage server.
  • the storage server may store the encoded video data and may transmit the encoded video data to the decoding device 120, such as a web server (eg, for a website), a File Transfer Protocol (FTP) server, etc.
  • FTP File Transfer Protocol
  • the encoding device 110 includes a video encoder 112 and an output interface 113.
  • the output interface 113 may include a modulator/demodulator (modem) and/or a transmitter.
  • the encoding device 110 may include a video source 111 in addition to the video encoder 112 and the input interface 113 .
  • Video source 111 may include at least one of a video capture device (eg, a video camera), a video archive, a video input interface for receiving video data from a video content provider, a computer graphics system Used to generate video data.
  • a video capture device eg, a video camera
  • a video archive e.g., a video archive
  • video input interface for receiving video data from a video content provider
  • computer graphics system Used to generate video data.
  • the video encoder 112 encodes the video data from the video source 111 to generate a code stream.
  • Video data may include one or more images (pictures) or sequence of pictures (sequence of pictures).
  • the code stream contains the encoding information of an image or image sequence in the form of a bit stream.
  • Encoded information may include encoded image data and associated data.
  • the associated data may include sequence parameter set (SPS), picture parameter set (PPS) and other syntax structures.
  • SPS sequence parameter set
  • PPS picture parameter set
  • An SPS can contain parameters that apply to one or more sequences.
  • a PPS can contain parameters that apply to one or more images.
  • a syntax structure refers to a collection of zero or more syntax elements arranged in a specified order in a code stream.
  • the video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113 .
  • the encoded video data can also be stored on a storage medium or storage server for subsequent reading by the decoding device 120 .
  • decoding device 120 includes input interface 121 and video decoder 122.
  • the decoding device 120 may also include a display device 123.
  • the input interface 121 includes a receiver and/or a modem. Input interface 121 may receive encoded video data over channel 130.
  • the video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .
  • the display device 123 displays the decoded video data.
  • Display device 123 may be integrated with decoding device 120 or external to decoding device 120 .
  • Display device 123 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • plasma display a plasma display
  • OLED organic light emitting diode
  • Figure 1 is only an example, and the technical solutions of the embodiments of the present application are not limited to Figure 1.
  • the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.
  • FIG. 2 is a schematic block diagram of a video encoder related to an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression of images (lossy compression), or can also be used to perform lossless compression (lossless compression) of images.
  • the lossless compression can be visually lossless compression (visually lossless compression) or mathematically lossless compression (mathematically lossless compression).
  • the video encoder 200 can be applied to image data in a luminance-chrominance (YCbCr, YUV) format.
  • YUV ratio can be 4:2:0, 4:2:2 or 4:4:4, Y represents brightness (Luma), Cb(U) represents blue chroma, Cr(V) represents red chroma, U and V represent Chroma, which is used to describe color and saturation.
  • 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr)
  • 4:2:2 means that every 4 pixels have 4 luminance components and 4 Chroma component (YYYYCbCrCbCr)
  • 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).
  • the video encoder 200 reads video data, and for each frame of image in the video data, divides one frame of image into several coding tree units (coding tree units, CTU).
  • CTB may be called “Tree block", “Largest Coding unit” (LCU for short) or “coding tree block” (CTB for short).
  • LCU Large Coding unit
  • CTB coding tree block
  • Each CTU can be associated with an equal-sized block of pixels within the image.
  • Each pixel can correspond to one luminance (luminance or luma) sample and two chrominance (chrominance or chroma) samples. Therefore, each CTU can be associated with one block of luma samples and two blocks of chroma samples.
  • a CTU size is, for example, 128 ⁇ 128, 64 ⁇ 64, 32 ⁇ 32, etc.
  • a CTU can be further divided into several coding units (Coding Units, CUs) for encoding.
  • CUs can be rectangular blocks or square blocks.
  • CU can be further divided into prediction unit (PU for short) and transform unit (TU for short), thus enabling coding, prediction, and transformation to be separated and processing more flexible.
  • the CTU is divided into CUs in a quad-tree manner, and the CU is divided into TUs and PUs in a quad-tree manner.
  • Video encoders and video decoders can support various PU sizes. Assuming that the size of a specific CU is 2N ⁇ 2N, the video encoder and video decoder can support a PU size of 2N ⁇ 2N or N ⁇ N for intra prediction, and support 2N ⁇ 2N, 2N ⁇ N, N ⁇ 2N, N ⁇ N or similar sized symmetric PU for inter prediction. The video encoder and video decoder can also support 2N ⁇ nU, 2N ⁇ nD, nL ⁇ 2N and nR ⁇ 2N asymmetric PUs for inter prediction.
  • the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform/quantization unit 230, an inverse transform/quantization unit 240, a reconstruction unit 250, and a loop filter unit. 260. Decode the image cache 270 and the entropy encoding unit 280. It should be noted that the video encoder 200 may include more, less, or different functional components.
  • the current block may be called the current coding unit (CU) or the current prediction unit (PU), etc.
  • the prediction block may also be called a predicted image block or an image prediction block
  • the reconstructed image block may also be called a reconstruction block or an image reconstructed image block.
  • prediction unit 210 includes inter prediction unit 211 and intra estimation unit 212. Since there is a strong correlation between adjacent pixels in a video frame, the intra-frame prediction method is used in video encoding and decoding technology to eliminate the spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in the video, the interframe prediction method is used in video coding and decoding technology to eliminate the temporal redundancy between adjacent frames, thereby improving coding efficiency.
  • the inter-frame prediction unit 211 can be used for inter-frame prediction.
  • Inter-frame prediction can include motion estimation (motion estimation) and motion compensation (motion compensation). It can refer to image information of different frames.
  • Inter-frame prediction uses motion information to find a reference from a reference frame. block, a prediction block is generated based on the reference block to eliminate temporal redundancy; the frames used in inter-frame prediction can be P frames and/or B frames, P frames refer to forward prediction frames, and B frames refer to bidirectional predictions frame.
  • Inter-frame prediction uses motion information to find reference blocks from reference frames and generate prediction blocks based on the reference blocks.
  • the motion information includes the reference frame list where the reference frame is located, the reference frame index, and the motion vector.
  • the motion vector can be in whole pixels or sub-pixels.
  • the reference frame found according to the motion vector is A block of whole pixels or sub-pixels is called a reference block.
  • Some technologies will directly use the reference block as a prediction block, and some technologies will process the reference block to generate a prediction block. Reprocessing to generate a prediction block based on a reference block can also be understood as using the reference block as a prediction block and then processing to generate a new prediction block based on the prediction block.
  • the intra-frame estimation unit 212 only refers to the information of the same frame image and predicts the pixel information in the current coded image block to eliminate spatial redundancy.
  • the frames used in intra prediction may be I frames.
  • Intra-frame prediction has multiple prediction modes. Taking the international digital video coding standard H series as an example, the H.264/AVC standard has 8 angle prediction modes and 1 non-angle prediction mode, and H.265/HEVC has been extended to 33 angles. prediction mode and 2 non-angle prediction modes.
  • the intra-frame prediction modes used by HEVC include planar mode (Planar), DC and 33 angle modes, for a total of 35 prediction modes.
  • the intra-frame modes used by VVC include Planar, DC and 65 angle modes, for a total of 67 prediction modes.
  • Residual unit 220 may generate a residual block of the CU based on the pixel block of the CU and the prediction block of the PU. For example, residual unit 220 may generate a residual block of a CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the PU of the CU. Predict the corresponding sample in the block.
  • Transform/quantization unit 230 may quantize the transform coefficients. Transform/quantization unit 230 may quantize transform coefficients associated with the TU of the CU based on quantization parameter (QP) values associated with the CU. Video encoder 200 may adjust the degree of quantization applied to transform coefficients associated with the CU by adjusting the QP value associated with the CU.
  • QP quantization parameter
  • Inverse transform/quantization unit 240 may apply inverse quantization and inverse transform to the quantized transform coefficients, respectively, to reconstruct the residual block from the quantized transform coefficients.
  • Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to produce a reconstructed image block associated with the TU. By reconstructing blocks of samples for each TU of a CU in this manner, video encoder 200 can reconstruct blocks of pixels of the CU.
  • the loop filtering unit 260 is used to process the inversely transformed and inversely quantized pixels to compensate for distortion information and provide a better reference for subsequent encoding of pixels. For example, a deblocking filtering operation can be performed to reduce the number of pixel blocks associated with the CU. block effect.
  • the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive compensation/adaptive loop filtering (SAO/ALF) unit, where the deblocking filtering unit is used to remove blocking effects, and the SAO/ALF unit Used to remove ringing effects.
  • SAO/ALF sample adaptive compensation/adaptive loop filtering
  • Decoded image cache 270 may store reconstructed pixel blocks.
  • Inter prediction unit 211 may perform inter prediction on PUs of other images using reference images containing reconstructed pixel blocks.
  • intra estimation unit 212 may use the reconstructed pixel blocks in decoded image cache 270 to perform intra prediction on other PUs in the same image as the CU.
  • Entropy encoding unit 280 may receive the quantized transform coefficients from transform/quantization unit 230 . Entropy encoding unit 280 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy encoded data.
  • Figure 3 is a schematic block diagram of a video decoder related to an embodiment of the present application.
  • the video decoder 300 includes an entropy decoding unit 310 , a prediction unit 320 , an inverse quantization/transformation unit 330 , a reconstruction unit 340 , a loop filtering unit 350 and a decoded image cache 360 . It should be noted that the video decoder 300 may include more, less, or different functional components.
  • Video decoder 300 can receive the code stream.
  • Entropy decoding unit 310 may parse the codestream to extract syntax elements from the codestream. As part of parsing the code stream, the entropy decoding unit 310 may parse entropy-encoded syntax elements in the code stream.
  • the prediction unit 320, the inverse quantization/transformation unit 330, the reconstruction unit 340 and the loop filtering unit 350 may decode the video data according to the syntax elements extracted from the code stream, that is, generate decoded video data.
  • prediction unit 320 includes inter prediction unit 321 and intra estimation unit 322.
  • Intra estimation unit 322 may perform intra prediction to generate predicted blocks for the PU. Intra estimation unit 322 may use an intra prediction mode to generate predicted blocks for a PU based on pixel blocks of spatially neighboring PUs. Intra estimation unit 322 may also determine the intra prediction mode of the PU based on one or more syntax elements parsed from the codestream.
  • the inter prediction unit 321 may construct a first reference image list (List 0) and a second reference image list (List 1) according to syntax elements parsed from the code stream. Additionally, if the PU uses inter-prediction encoding, entropy decoding unit 310 may parse the motion information of the PU. Inter prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. Inter prediction unit 321 may generate a predictive block for the PU based on one or more reference blocks of the PU.
  • Inverse quantization/transform unit 330 may inversely quantize (ie, dequantize) transform coefficients associated with a TU. Inverse quantization/transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.
  • inverse quantization/transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to produce a residual block associated with the TU.
  • Reconstruction unit 340 uses the residual blocks associated with the TU of the CU and the prediction blocks of the PU of the CU to reconstruct the pixel blocks of the CU. For example, reconstruction unit 340 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.
  • Loop filtering unit 350 may perform deblocking filtering operations to reduce blocking artifacts for blocks of pixels associated with the CU.
  • Video decoder 300 may store the reconstructed image of the CU in decoded image cache 360 .
  • the video decoder 300 may use the reconstructed image in the decoded image cache 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.
  • the basic process of video encoding and decoding is as follows: at the encoding end, an image frame is divided into blocks.
  • the prediction unit 210 uses intra prediction or inter prediction to generate a prediction block of the current block.
  • the residual unit 220 may calculate a residual block based on the prediction block and the original block of the current block, that is, the difference between the prediction block and the original block of the current block.
  • the residual block may also be called residual information.
  • the residual block undergoes transformation and quantization processes such as transformation/quantization unit 230 to remove information that is insensitive to human eyes to eliminate visual redundancy.
  • the residual block before transformation and quantization by the transformation/quantization unit 230 may be called a time domain residual block, and the time domain residual block after transformation and quantization by the transformation/quantization unit 230 may be called a frequency residual block. or frequency domain residual block.
  • the entropy encoding unit 280 receives the quantized change coefficient output from the change quantization unit 230, and may perform entropy encoding on the quantized change coefficient to output a code stream. For example, the entropy encoding unit 280 may eliminate character redundancy according to the target context model and probability information of the binary code stream.
  • the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block.
  • the prediction unit 320 uses intra prediction or inter prediction for the current block based on the prediction information to generate a prediction block of the current block.
  • the inverse quantization/transform unit 330 uses the quantization coefficient matrix obtained from the code stream to perform inverse quantization and inverse transformation on the quantization coefficient matrix to obtain a residual block.
  • the reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstruction block.
  • the reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or based on the blocks to obtain a decoded image.
  • the encoding end also needs similar operations as the decoding end to obtain the decoded image.
  • the decoded image may also be called a reconstructed image, and the reconstructed image may be used as a reference frame for inter-frame prediction for
  • the block division information determined by the encoding end as well as mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., are carried in the code stream when necessary.
  • the decoding end determines the same block division information as the encoding end by parsing the code stream and analyzing the existing information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information, thereby ensuring the decoded image obtained by the encoding end It is the same as the decoded image obtained by the decoding end.
  • the current block can be the current coding unit (CU) or the current prediction unit (PU), etc.
  • the general hybrid coding framework will first perform prediction, and the prediction uses the correlation performance in space or time to obtain an image that is the same or similar to the current block.
  • the prediction uses the correlation performance in space or time to obtain an image that is the same or similar to the current block.
  • the hybrid coding framework will subtract the predicted image from the original image of the current block to obtain the residual image, or the current block minus the predicted block will obtain the residual block.
  • Residual blocks are usually much simpler than the original image, so prediction can significantly improve compression efficiency.
  • the residual block is not directly encoded, but is usually transformed first. Transformation is to transform the residual image from the spatial domain to the frequency domain and remove the correlation of the residual image. After the residual image is transformed into the frequency domain, since the energy is mostly concentrated in the low-frequency region, most of the transformed non-zero coefficients are concentrated in the upper left corner. Next, quantization is used for further compression. And since the human eye is not sensitive to high frequencies, a larger quantization step size can be used in high-frequency areas.
  • the current common transformation method is DCT-II.
  • DCT-II cannot achieve optimal transformation. Therefore, multiple transformation kernels are designed to adaptively select the optimal transformation kernel for different image blocks, which is closer to the theoretical optimum. Excellent transformation performance.
  • MTS Multiple Transform Selection
  • N is the number of points of the original signal
  • ⁇ 0 is the compensation coefficient
  • MTS image sequence level
  • Both length and width are less than or equal to 32;
  • the current TU is in non-transform skip mode
  • the transformation method is selected from the candidate set composed of the combination of DST-VII and DCT-VIII.
  • the candidate set is shown in Table 2 below, and the rate distortion costs RD2, RD3... are calculated.
  • An implementation of an encoder using MTS is as follows. First, the input image is divided into multiple non-overlapping CTU blocks. Then, each CTU is processed in sequence according to the raster scanning order, and the CTU is divided into several CUs according to the prediction cost.
  • An implementation of a decoder using MTS is as follows.
  • the input code stream undergoes entropy decoding, inverse quantization, and inverse transformation.
  • the inverse transformation process mainly includes the following steps: (1) First, derive the transformation type trType based on code stream information such as sps_mts_enabled_flag. (2) Derive the transformation matrix transMatrix according to the transformation type trType and CU size. (3) Use the transformation matrix transMatrix and the transformation coefficient to calculate the inverse transformation value to obtain the prediction residual value, and reconstruct the image based on the residual block. Finally, the reconstructed image is sent to the DBF/SAO/ALF filter, and the filtered image is sent to the buffer to wait for video playback.
  • the essence of MTS is to select the deformation body of DCT (DCT-VIII, DST-VII) for transformation.
  • DCT-VIII, DST-VII DCT-VIII, DST-VII
  • the premise to ensure the effectiveness of DCT is that the signal energy of the image block is concentrated, so that the high-frequency components can be transformed after DCT transformation. Concentrated in the upper left corner, most areas in the lower right corner are low-frequency components. In other words, only for images with concentrated signal energy, DCT transformation can achieve better compression performance. On the contrary, when the signal energy is not concentrated, the DCT transformation performance is poor, so the performance gain brought by choosing the best among the poor is very limited.
  • the embodiment of the present application first determines the continuity parameter of the image area where the current coding block is located.
  • the continuity parameter is used to indicate the continuity of changes in pixels in the image area, where the continuity of changes in pixels can also be Understood as the correlation of pixels. Due to the poor continuity of the image area, it means that the signal energy is not concentrated, and the DCT transformation effect is poor. At this time, there is no need to spend a lot of computing resources to select the optimal transformation method from multiple transformation methods. Based on this, the embodiment of the present application determines the transformation method of the current coding block according to the continuity parameter of the image area.
  • DTC- II method for transformation there is no need to select the optimal transformation method from multiple transformation methods for transformation, thus avoiding unnecessary calculation overhead and improving coding speed.
  • FIG. 4 is a schematic flowchart of an image encoding method provided by an embodiment of the present application. The embodiment of the present application is applied to the video encoder shown in FIGS. 1 and 2 .
  • the method in the embodiment of this application includes:
  • This continuity parameter is used to indicate the continuity of changes in pixels in the image area.
  • the encoder receives a video stream, which is composed of a series of image frames, and performs video encoding on each frame of image in the video stream.
  • the video encoder divides the image frames into blocks to obtain the current encoding block.
  • the current coding block is also called a current block, a current image block, a coding block, a current coding unit, a current block to be coded, a current image block to be coded, etc.
  • the blocks divided by the traditional method contain both the chrominance component of the current coding block position and the luminance component of the current coding block position.
  • the separation tree technology can be divided into separate component blocks, such as separate brightness blocks and separate chroma blocks.
  • the brightness block can be understood as containing only the brightness component of the current coding block position
  • the chroma block can be understood as containing only the current coding block position. Encode the chroma component of the block position. In this way, luminance components and chrominance components at the same position can belong to different blocks, and the division can be more flexible. If the separation tree is used in CU partitioning, then some CUs contain both luminance and chrominance components, some CUs only contain luminance components, and some CUs only contain chrominance components.
  • the current coding block in the embodiment of the present application only includes chrominance components, which can be understood as chrominance blocks.
  • the current coding block in the embodiment of the present application only includes the luminance component, which can be understood as a luminance block.
  • the current coding block includes both luma and chrominance components.
  • the encoding process in this embodiment of the present application is to divide the current image frame into blocks to obtain the current coding block, and use the intra-frame or inter-frame prediction method to predict the current coding block to obtain the prediction value of the current coding block.
  • the original value of the current coding block is subtracted from the predicted value to obtain the residual value of the current coding block.
  • Determine the transformation method corresponding to the current coding block use this transformation method to transform the residual value, and obtain the transformation coefficient.
  • the transform coefficients are quantized, and the quantized transform coefficients are encoded to obtain a code stream.
  • the embodiment of the present application involves the transformation process in the above encoding process.
  • adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods can improve the transformation effect.
  • the compression effect brought by the transformation is poor.
  • adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods to transform these images whose signal energy is not concentrated not only has an insignificant compression effect, but also generates unnecessary computational overhead and increases encoding time.
  • the current coding block before transforming the current coding block, it first determines whether the signal energy in the image area where the current coding block is located is concentrated. For an image, if the changes in pixel points in the image are discontinuous, it means that the spatial correlation of the image is weak and the signal energy is not concentrated. If the changes in pixels in the image are continuous, it means that the spatial correlation of the image is strong and the signal energy is concentrated.
  • the embodiment of the present application uses the size of the continuity parameter of the image area to indicate whether the signal energy of the image area is concentrated. For example, if the continuity parameter of the image area is large, it indicates the intensity of the pixels in the image area. The change is continuous, which indicates that the signal energy in this image area is concentrated. If the continuity parameter of the image area is small, it indicates that the changes in pixel points in the image area are discontinuous, which further indicates that the signal energy in the image area is not concentrated. For image areas where signal energy is not concentrated, selecting the optimal transformation method from multiple transformation methods will not only fail to bring significant transformation effects, but also generate unnecessary computational overhead and increase encoding time. Therefore, in the embodiment of the present application, For image areas where signal energy is not concentrated, skip the operation of selecting the optimal transformation method from multiple transformation methods (such as skipping MTS technology) to save unnecessary computing overhead and improve coding efficiency.
  • the continuity parameter of the image area is large, it indicates the intensity of the pixels in the image area. The change is continuous, which indicates that the signal energy in this image area is concentrated
  • this embodiment of the present application first determines the continuity parameter of the image area where the current coding block is located.
  • the continuity parameter is used to indicate the continuity of changes in pixels in the image area.
  • the transformation method of the current coding block is determined. For example, if the continuity parameter is small, the default first transformation method is used as the transformation method of the current coding block, and the selection of the best transformation method from multiple transformation methods is skipped. Optimize transformation operations to save unnecessary computing overhead and improve coding efficiency.
  • the embodiment of the present application does not limit the size of the image area, and it can be any image area including the current coding block.
  • the image area is the current image frame
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixel points in the current image frame.
  • the image area is the current image sequence
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image sequence. In this way, the embodiment of the present application can determine the transformation method of any coding block in the current image sequence based on the continuity parameter of the current image sequence.
  • the image area is the current image patch
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image patch. In this way, the embodiment of the present application can determine the transformation method of any coding block in the current image slice according to the continuity parameter of the current image slice.
  • the image area is the current coding block
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current coding block.
  • the methods for determining the continuity parameters of the image area include but are not limited to the following:
  • Method 1 is to determine the continuity parameters of the image area through the gradient map, that is, the above S401 includes the following steps of S401-A1 and S401-A2:
  • S401-A2 Determine the continuity parameter of the image area according to the gradient map of the gradient map.
  • the gradient map of the image area can reflect the change trend of the pixels in the image area, and the gradient map of the gradient map of the image area can reflect the change rate of the pixels in the image area, that is, the continuity of change.
  • the continuity parameter of the image area is determined by calculating the gradient map of the gradient map of the image area.
  • the gradient map Gmap of the image area is determined according to the following formula (1):
  • I(x, y) is the pixel value of image area I at position (x, y)
  • I(x+1, y) is the pixel value of image area I at position (x+1, y)
  • I(x, y+1) is the pixel value of image area I at position (x, y+1)
  • Gmap(x, y) is the gradient value of the pixel point at position (x, y) in image area I .
  • the gradient map GGmap of the gradient map of the image area is determined according to the following formula (2):
  • GGmap(x,y)
  • Gmap(x+1,y) is the gradient value of the pixel at position (x+1,y) in image area I
  • Gmap(x,y+1) is the position (x,y+ The gradient value of the pixel at 1)
  • Gmap(x+1,y) and Gmap(x,y+1) can be determined according to the above formula (1)
  • GGmap(x,y) is the gradient value of the gradient value of the pixel at position (x,y) in image area I.
  • the above formula (1) and formula (2) show that the gradient value and the gradient value of the pixel point (x, y) of the image area I are determined. Other pixel points in the image area I refer to the pixel point (x, y). ) method, determine the gradient value of each pixel in the image area I, and obtain the gradient map Gmap of the image area I. Use the above formula (2) to calculate the gradient value corresponding to each pixel in the gradient map Gmap of the image area I, and then obtain the gradient map GGmap of the gradient map of the image area I.
  • the above formula (1) and formula (2) are just examples.
  • the embodiments of the present application can also use other methods to determine the gradient map of the image area and the gradient map of the gradient map.
  • the above formula ( 1) and formula (2) are deformed, and the gradient map is determined using the deformed formula, as well as the gradient map of the gradient map.
  • the continuity parameter of the image area is determined based on the gradient map GGmap of the gradient map.
  • the embodiment of the present application does not limit the method of determining the continuity parameter of the image area based on the gradient map of the gradient map in S401-A2 above.
  • the median value of each gradient value in the gradient map of the image region is determined as the continuity parameter of the image region.
  • the average value of the gradient values in the gradient map of the image area is determined as the continuity parameter of the image area.
  • the above-mentioned method 1 determines the continuity parameter of the image area by determining the gradient map of the gradient map of the image area.
  • the calculation method is simple and the calculation cost is small, thereby improving the coding efficiency.
  • the following method two can also be used to determine the continuity parameter of the image area.
  • the second method is to determine the continuity parameters of the image area by determining the spatial redundancy of the image area. That is, the above S401 includes the following steps of S401-B1 and S401-B2:
  • S401-B2 Determine the continuity parameter of the image area according to the residual value of the image area, where the residual value is negatively correlated with the continuity parameter.
  • the spatial redundancy of the image area is small, it means that the correlation of the image area is strong, the changes of pixel points in the image area are more continuous, and the signal energy of the image area is more concentrated. If the spatial redundancy of the image area is large, it means that the correlation of the image area is weak, the changes of pixel points in the image area are discontinuous, and the signal energy of the image area is not concentrated.
  • the spatial redundancy of the image area can be represented by the residual value of the image area, that is, a large residual value indicates that the spatial redundancy of the image area is large, and a small residual value indicates that the spatial redundancy of the image area is small.
  • intra-frame prediction is used to calculate the spatial redundancy of the image. Therefore, the embodiment of the present application uses the intra-frame prediction method to calculate the predicted value of the image area, and calculates the difference between the original value and the predicted value of the image area. Determine the residual value of this image area.
  • the residual value of the image area may be a superposition of the residual values of each coding block included in the image area.
  • the image area includes N coding blocks, and N is a positive integer.
  • Intra-frame prediction is used to determine the predicted value of each coding block.
  • the original value of the coding block is subtracted from the predicted value, and we get The residual value of this coding block.
  • the residual values of N coding blocks are superimposed to obtain the residual value of the image area.
  • the continuity parameter of the image area is determined.
  • the residual value of the image area is negatively correlated with the continuity parameter. That is, the greater the residual value of the image area, the greater the continuity parameter of the image area. The smaller the value, the smaller the residual value of the image area, and the greater the continuity parameter of the image area.
  • the embodiment of the present application does not limit the method of determining the continuity parameter of the image area based on the residual value of the image area.
  • the reciprocal of the residual value of the image area is determined as the continuity parameter of the image area. For example, if the residual value of the image area is a, then the continuity parameter of the image area is determined to be 1. /a.
  • the difference between the preset value and the residual value of the image area is determined as the continuity parameter of the image area.
  • the preset value is a positive number, such as the preset value.
  • the value is 1.
  • the continuity parameter of the image area is determined to be 1-a.
  • step S402 is performed to determine the transformation method of the current coding block based on the continuity parameter of the image area.
  • S402. Determine the transformation method of the current coding block according to the continuity parameter of the image area.
  • the continuity parameter of the image area is used to indicate the continuity of changes in the pixels in the image area. If the continuity parameter of the image area is small, it means that the signal energy in the image area is not concentrated. In this case, DTC is used The transformation cannot bring significant compression effect. In this case, calculating the rate distortion cost of multiple transformation methods to select the optimal transformation method from these multiple transformation methods will not only not bring beneficial compression effects, but also The computational cost is high. Therefore, when the continuity parameter of the image area is large, the operation of selecting the optimal transformation method from multiple transformation methods is skipped to save computing resources and improve coding efficiency.
  • the methods for determining the transformation method of the current coding block according to the continuity parameter of the image area include but are not limited to the following:
  • Method 1 If the continuity parameter of the image area is less than the preset threshold, it is determined that the transformation mode of the current coding block is the first transformation mode.
  • the encoder skips the operation of selecting the optimal transform scheme from multiple transform schemes, for example, skips the MTS operation, and directly determines the first transform scheme as the target transform scheme of the current coding block.
  • the above-mentioned first transformation mode is one of multiple preset transformation modes.
  • the first transformation mode is DTC-II mode, DST-VII mode and DCT-VIII mode. any transformation method.
  • the first conversion mode is the DTC-II mode.
  • the embodiment of the present application does not place a limit on the preset threshold, which is determined based on actual needs.
  • Method 2 If the continuity parameter of the image area is greater than or equal to the preset threshold, the transformation method of the current coding block is determined from M candidate transformation methods, where M is a positive integer greater than 1.
  • the encoder performs an operation of selecting the optimal transform method from M transform methods, such as performing an MTS operation, to determine the transform method of the current coding block from these M candidate transform methods.
  • This embodiment of the present application does not limit the number and types of M candidate transformation methods.
  • the M candidate transformation modes include transformation modes composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode. Since the images are all 2-dimensional, the amount of computation and memory overhead of directly performing two-dimensional transformation were unacceptable to the hardware conditions at the time. Therefore, when using the above-mentioned DCT-II, DCT-VIII, and DST-VII in the standard for transformation, both are split into horizontal and vertical directions, and a two-step one-dimensional transformation is performed. For example, perform horizontal transformation first and then vertical transformation, or perform vertical transformation first and then horizontal transformation.
  • the M candidate transformation modes in the embodiment of the present application can be any combination of DTC-II mode, DST-VII mode and DCT-VIII mode in the horizontal and vertical directions.
  • candidate transformation method 1 is DTC-II in the horizontal direction and DST-VII in the vertical direction
  • candidate transformation method 2 is DTC-II in the horizontal direction and DCT-VIII in the vertical direction
  • candidate transformation method 3 is The horizontal direction is DST-VII mode and the vertical direction is DTC-II mode.
  • the horizontal direction of candidate transformation method 4 is DST-VII mode and the vertical direction is DCT-VIII mode.
  • the horizontal direction of candidate transformation method 5 is DCT- The VIII mode and the vertical direction are DTC-II mode.
  • the horizontal direction and vertical direction of candidate transformation mode 6 are DCT-VIII mode and the vertical direction are DST-VII mode.
  • the horizontal and vertical modes of candidate transformation mode 7 are both DTC-II mode
  • the horizontal direction and the vertical mode of candidate transformation mode 8 are both DST-VII mode
  • the horizontal direction and vertical mode of candidate transformation mode 9 are both DCT-VIII mode.
  • the M candidate transformation modes include multiple transformation modes corresponding to MTS, such as the DTC-II mode, and the combined transformation modes shown in Table 2 above.
  • determining the transformation mode of the current coding block from M candidate transformation modes includes the following steps:
  • Step 1 Determine the rate distortion cost when using M candidate transformation methods to transform the current coding block
  • Step 2 Select a candidate transformation method from the M candidate transformation methods as the transformation method of the current coding block according to the rate distortion cost corresponding to the M candidate transformation methods.
  • the method of determining the transformation mode of the current coding block from the M candidate transformation modes may be: first, use the first candidate transformation mode among the M candidate transformation modes to transform the current coding block. , and calculate the rate distortion cost corresponding to the first candidate transformation method. If the rate distortion cost corresponding to the first candidate transformation method is less than or equal to the preset value, the first candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the first candidate transformation method is greater than the preset value, use the second candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the corresponding rate distortion cost of the second candidate transformation method. rate distortion cost.
  • the second candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the second candidate transformation method is greater than the preset value, repeat the above steps, use the third candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the third candidate transformation method The rate distortion cost corresponding to the transformation method.
  • a candidate transformation method is selected from M candidate transformation methods as the transformation method of the current coding block.
  • the method of determining the transformation mode of the current coding block from M candidate transformation modes may be: using these M candidate transformation modes to transform the current coding block respectively, and calculating the M candidates The rate distortion cost corresponding to each candidate transformation method in the transformation method.
  • the candidate transformation mode with the smallest rate distortion cost is determined as the transformation mode of the current coding block.
  • the encoder determines the transformation method of the current coding block according to the continuity parameter of the image area.
  • the encoder may also use the following method 3 to determine the transformation mode of the current coding block.
  • the encoder first determines a flag based on the continuity parameter of the image area, and determines the transformation mode of the current coding block based on the flag. That is, the above S402 includes the following steps of S402-A1 and S402-A2:
  • the first flag is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes, where M is a positive integer greater than 1.
  • the first flag is used to indicate whether the image area is allowed to use the MTS technology, or the first flag is used to indicate the image area. Whether to skip MTS operations.
  • the method of determining the first mark according to the continuity parameter of the image area in the above S402-A1 includes the following examples:
  • Example 1 If the continuity parameter of the image area is less than the preset threshold, it means that the signal energy in the image area is not concentrated. At this time, the value of the first flag is determined to be the first value. The first value is used to indicate that the image area is not allowed. The transformation method is determined from M candidate transformation methods.
  • Example 2 if the continuity parameter of the image area is greater than or equal to the preset threshold, it means that the signal energy in the image area is concentrated. At this time, the value of the first flag is determined to be the second value. The second value is used to indicate that the image area is allowed to change from The transformation method is determined among M candidate transformation methods.
  • the first value is not equal to the second value.
  • the first value is 0.
  • the second value is 1.
  • the transformation mode of the current coding block is determined based on the first flag.
  • the value of the first flag is the first value, it indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes. At this time, the step of determining the transformation mode from the M candidate transformation modes is skipped. operation, but directly determines the default first transformation mode as the transformation mode of the current coding block.
  • the value of the first flag is the second value, it indicates that the image area is allowed to determine the transformation mode from M candidate transformation modes. At this time, the transformation mode of the current coding block is determined from the M candidate transformation modes.
  • the current coding block is transformed using the first candidate transformation method among the M candidate transformation methods, and the rate distortion cost corresponding to the first candidate transformation method is calculated. If the rate distortion cost corresponding to the first candidate transformation method is less than or equal to the preset value, the first candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the first candidate transformation method is greater than the preset value, use the second candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the corresponding rate distortion cost of the second candidate transformation method. rate distortion cost.
  • a candidate transformation method is selected from M candidate transformation methods as the transformation method of the current coding block.
  • the M candidate transformation methods are used to transform the current coding block respectively, and the rate distortion cost corresponding to each of the M candidate transformation methods is calculated.
  • the candidate transformation mode with the smallest rate distortion cost is determined as the transformation mode of the current coding block.
  • the encoder determines the transformation mode of the current coding block according to the above method, it performs the following steps of S403.
  • the current coding block is predicted using the intra prediction method and/or the inter prediction method to obtain the prediction value of the current coding block. Then, based on the prediction value of the current coding block and the current coding block, the prediction value of the current coding block is obtained. Residual value. For example, subtract the predicted value from the pixel value of the current coding block to obtain the residual value of the current coding block.
  • the transformation method of the current coding block determined according to the above method is the DCT-II transformation method
  • the DCT-II transformation method is used to transform the residual of the current coding block Transform the value to obtain the transformation coefficient of the current coding block.
  • the transformation method of the current coding block determined according to the above method is the DCT-VIII transformation method, then according to the formula shown in Table 2 above, the DCT-VIII transformation method is used to transform the residual of the current coding block. The difference value is transformed to obtain the transformation coefficient of the current coding block.
  • the transformation method of the current coding block determined according to the above method is the DST-VII transformation method
  • the DST-VII transformation method is used to transform the residual of the current coding block.
  • the difference value is transformed to obtain the transformation coefficient of the current coding block.
  • the encoder writes the above-mentioned first flag into the code stream, so that the decoder determines the transform type based on the first flag and implements inverse transformation of the transform coefficients.
  • the code stream also includes a second flag.
  • the second flag is used to indicate whether the current picture sequence is allowed to be copied from The transformation method is determined among M candidate transformation methods. In the MTS technology, the second flag is specifically used to indicate whether the current image sequence allows the use of the MTS technology. If the second flag indicates that the current image sequence is not allowed to use the MTS technology, it is determined that all coded image blocks under the current image sequence do not use the MTS technology, and the first transformation method is determined as the target transformation method to calculate the residual value. Transform to get the transformation coefficients. If the second flag indicates that the current image sequence is allowed to use the MTS technology, then all coded image blocks under the current image sequence use the MTS technology to determine the transformation method.
  • the first flag multiplexes the second flag if the image area is the current image sequence.
  • the embodiment of the present application does not limit the specific positions of the first mark and the second mark in the code stream.
  • pic_mts_enable_flag can be used to represent the first flag.
  • sps_explicit_mts_inter_enabled_flag may be used to represent the second flag.
  • the first flag and the second flag are located in the coding unit. At this time, part of the syntax of the coding unit is as shown in Table 3:
  • a constraint first flag is added to the existing coding unit syntax, and the first flag is used to indicate whether the current area skips the MTS operation.
  • the decoding end can determine the transformation type trType of the current coding block according to the first flag during decoding.
  • the image coding method provided by the embodiment of the present application determines the continuity parameter of the image area where the current coding block is located.
  • the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area; based on the continuity parameter of the image area, the current coding block is determined based on the continuity parameter of the image area.
  • the default first transformation method can be used as The transformation method of the current coding block, and skips the operation of selecting the optimal transformation method from multiple transformation methods to save unnecessary computing overhead and improve coding efficiency.
  • Figure 5 is a schematic flowchart of an image encoding method provided by an embodiment of the present application.
  • Figure 5 can be understood as a more specific encoding method of the encoding method shown in Figure 4.
  • the encoding method according to the embodiment of the present application includes the following steps:
  • the image area is the current image frame as an example.
  • the continuity parameter is used to indicate the continuity of changes in pixels in the current image frame.
  • a gradient map of the current image frame is determined, and a gradient map of the gradient map is determined, and a continuity parameter of the current image frame is determined based on the gradient map of the gradient map. For example, the average value of the gradient values in the gradient map of the current image frame is determined as the continuity parameter of the current image frame.
  • a residual value of the current image frame is determined, and a continuity parameter of the current image frame is determined based on the residual value of the current image frame, where the residual value is negatively correlated with the continuity parameter.
  • the current image frame skips the MTS operation, and sets the value of the first flag to the first value.
  • the embodiment of this application takes MTS technology as an example.
  • the current image frame skips the MTS operation. That is to say, all coding blocks under the current image frame do not use MTS technology to determine the transformation method.
  • the default first transformation mode is determined as the transformation mode of all coding blocks in the current image frame.
  • pic_mts_enable_flag is used to represent the first flag.
  • the first value is 0.
  • the first transformation mode is the DCT-II mode.
  • the continuity parameter of the current image frame is greater than or equal to the preset threshold, it means that the signal energy of the current image frame is concentrated.
  • the current image frame is allowed to perform MTS operations. At this time, when determining the specific transformation method for the coding block in the current image frame, MTS technology can be used to determine the optimal transformation method.
  • pic_mts_enable_flag is used to represent the first flag.
  • the second value is 1.
  • the DCT-II method is directly selected as the transformation method of the current coding block.
  • the transform coefficients of the current transform block are directly encoded without quantization to obtain a code stream.
  • the transform coefficient of the current encoding block is quantized to obtain the quantized coefficient, and the quantized coefficient is encoded to obtain the code stream.
  • the encoder writes the above-mentioned first flag into the code stream, so that the decoder determines the transformation type corresponding to the inverse transformation according to the first flag.
  • the image coding method provided by the embodiment of the present application determines the continuity parameter of the current image frame, and compares the continuity parameter with the preset threshold. If the continuity parameter is less than the preset threshold, the MTS operation is skipped and the MTS operation is skipped.
  • the first transformation mode is determined as the transformation mode of the current coding block, thereby saving unnecessary calculation overhead and improving coding efficiency.
  • FIG. 6 is a schematic flowchart of a video decoding method provided by an embodiment of the present application.
  • the embodiment of the present application is applied to the decoders shown in FIGS. 1 and 3 .
  • the method in the embodiment of this application includes:
  • the current decoding block may also be referred to as the current block, the current decoding unit, the decoding block, the block to be decoded, the current block to be decoded, etc.
  • the currently decoded block when the currently decoded block includes a chroma component and does not include a luma component, the currently decoded block may be called a chroma block.
  • the currently decoded block when the currently decoded block includes a luma component and does not include a chrominance component, the currently decoded block may be called a luma block.
  • the above-mentioned transform coefficients are transform coefficients obtained by transforming the residual value of the current decoding block at the encoding end.
  • the decoding end decodes the code stream and obtains the transform coefficients of the current decoded block, including but not limited to the following:
  • Method 1 If the encoding end is encoding, the transform coefficients are not quantized, but the transform coefficients are directly encoded to obtain a code stream. In this way, the decoding end decodes the code stream and can directly obtain the transform coefficient of the current decoding block from the code stream.
  • Method 2 During encoding, the encoding end quantizes the transform coefficients to obtain quantized coefficients, and then encodes the quantized coefficients to obtain a code stream. In this way, the decoder decodes the code stream to obtain the quantized coefficients of the current decoded block, and performs inverse quantization on the quantized coefficients to obtain the transform coefficients of the current decoded block.
  • S602. Determine the first flag of the image area where the current coding block is located.
  • the above-mentioned first flag is used to determine the transform type of the current coding block.
  • the above-mentioned first flag is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes.
  • the first flag in the embodiment of the present application is determined based on the continuity parameter of the image area.
  • the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area.
  • adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods can improve the transformation effect.
  • the compression effect brought by the transformation is poor.
  • adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods to transform these images whose signal energy is not concentrated not only has an insignificant compression effect, but also generates unnecessary computational overhead and increases encoding time.
  • the embodiment of the present application uses the size of the continuity parameter of the image area to indicate whether the signal energy of the image area is concentrated. For example, if the continuity parameter of the image area is large, it indicates the intensity of the pixels in the image area. The change is continuous, which indicates that the signal energy in this image area is concentrated. If the continuity parameter of the image area is small, it indicates that the changes in pixel points in the image area are discontinuous, which further indicates that the signal energy in the image area is not concentrated. For image areas where signal energy is not concentrated, selecting the optimal transformation method from multiple transformation methods will not only fail to bring significant transformation effects, but also generate unnecessary computational overhead and increase encoding and decoding time. Therefore, in the embodiment of the present application, for image areas where signal energy is not concentrated, skip the operation of selecting the optimal transformation method from M candidate transformation methods (such as skipping MTS technology) to save unnecessary computing overhead.
  • M candidate transformation methods such as skipping MTS technology
  • the embodiment of the present application does not limit the size of the image area, and it can be any image area including the current decoding block.
  • the image area is the current image frame
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixel points in the current image frame.
  • the first flag is used to indicate whether the current image frame is allowed to determine the transformation mode from M candidate transformation modes.
  • the image area is the current image sequence
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image sequence.
  • the first flag is used to indicate whether the current image sequence is allowed to determine the transformation mode from M candidate transformation modes.
  • the image area is the current image patch
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image patch.
  • the first flag is used to indicate whether the current image slice is allowed to determine the transformation mode from M candidate transformation modes.
  • the image area is the current coding block
  • the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current coding block.
  • the first flag is used to indicate whether the current coding block is allowed to determine the transformation mode from M candidate transformation modes.
  • This embodiment of the present application does not limit the number and types of M candidate transformation methods.
  • the M candidate transformation modes include transformation modes composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode. Since the images are all 2-dimensional, the amount of computation and memory overhead of directly performing two-dimensional transformation were unacceptable to the hardware conditions at the time. Therefore, when using the above-mentioned DCT-II, DCT-VIII, and DST-VII in the standard for transformation, both are split into horizontal and vertical directions, and a two-step one-dimensional transformation is performed. For example, perform horizontal transformation first and then vertical transformation, or perform vertical transformation first and then horizontal transformation.
  • the M candidate transformation modes in the embodiment of the present application can be any combination of DTC-II mode, DST-VII mode and DCT-VIII mode in the horizontal and vertical directions.
  • candidate transformation method 1 is DTC-II in the horizontal direction and DST-VII in the vertical direction
  • candidate transformation method 2 is DTC-II in the horizontal direction and DCT-VIII in the vertical direction
  • candidate transformation method 3 is The horizontal direction is DST-VII mode and the vertical direction is DTC-II mode.
  • the horizontal direction of candidate transformation method 4 is DST-VII mode and the vertical direction is DCT-VIII mode.
  • the horizontal direction of candidate transformation method 5 is DCT- The VIII mode and the vertical direction are DTC-II mode.
  • the horizontal direction and vertical direction of candidate transformation mode 6 are DCT-VIII mode and the vertical direction are DST-VII mode.
  • the horizontal and vertical modes of candidate transformation mode 7 are both DTC-II mode
  • the horizontal direction and the vertical mode of candidate transformation mode 8 are both DST-VII mode
  • the horizontal direction and vertical mode of candidate transformation mode 9 are both DCT-VIII mode.
  • the M candidate transformation modes include multiple transformation modes corresponding to MTS, such as the DTC-II mode, and the combined transformation modes shown in Table 2 above.
  • the transform type trType when inversely transforming the transform coefficients of the current decoding block, the transform type trType needs to be determined first, and then the transform coefficients of the current decoding block are inversely transformed according to the transform type trType to obtain the residual value of the current decoding block.
  • the transformation type trType is determined according to the first flag. For example, when the first flag is 0, it means that the image area does not allow the optimal transformation method to be selected from M candidate transformation methods, and then the current decoding block can be determined.
  • the transformation type trType is 0.
  • the methods for determining the first mark include but are not limited to the following:
  • Method 1 The decoder determines the first flag based on the continuity parameter of the image area.
  • the above S602 includes the following steps S602-A1 and S602-A2, that is, first determines the continuity parameter of the image area, and then determines the first flag based on the continuity parameter of the image area. Parameters determine the first marker of the image area.
  • the decoding end determines the continuity parameter of the image area
  • the image area is not completely decoded.
  • the continuity parameters of adjacent decoded areas of the image area can be used as the continuity parameters of the image area.
  • the methods for determining the continuity parameters of the image area include but are not limited to the following examples:
  • Example 1 determine the continuity parameters of the image area through the gradient map, that is, the above S602-A1 includes the following steps from S602-A1-11 to S602-A1-13:
  • adjacent decoded regions are consistent in size with image regions.
  • adjacent decoded regions are not consistent in size with image regions.
  • the gradient map of an image can reflect the change trend of pixels in the image, and the gradient map of the image can reflect the rate of change of pixels in the image, that is, the continuity of change.
  • the continuity parameter of the adjacent decoded region can be determined by calculating the gradient map of the gradient map of the adjacent decoded region.
  • the gradient map Gmap' of the adjacent decoded area is determined according to the following formula (3):
  • I'(x, y) is the pixel value of the adjacent decoded area I' at the position (x, y)
  • I'(x+1, y) is the adjacent decoded area I' at the position (x +1, y)
  • I'(x, y+1) is the pixel value of the adjacent decoded area I' at position (x, y+1)
  • Gmap'(x, y) is the corresponding The gradient value of the pixel at position (x, y) in the adjacent decoded area I'.
  • the gradient map GGmap' of the gradient map of the adjacent decoded area is determined according to the following formula (4):
  • GGmap'(x,y)
  • Gmap'(x+1,y)' is the gradient value of the pixel at position (x+1,y) in the adjacent decoded area I'
  • Gmap(x,y+1)' is the adjacent decoded area I'.
  • the gradient value of the pixel at position (x, y+1) in area I', where Gmap'(x+1,y) and Gmap'(x,y+1) can be determined according to the above formula (3).
  • GGmap’ (x, y) is the gradient value of the gradient value of the pixel at position (x, y) in the adjacent decoded area I’.
  • the above formulas (3) and (4) show that the gradient value of the pixel point (x, y) in the adjacent decoded area I' and the gradient value of the gradient value are determined. Other pixels in the adjacent decoded area I' By referring to the pixel point (x, y), the gradient value of each pixel point in the adjacent decoded area I' is determined, and the gradient map Gmap' of the adjacent decoded area I' is obtained. Use the above formula (4) to calculate the gradient value corresponding to each pixel point in the gradient map Gmap' of the adjacent decoded area I', and then obtain the gradient map GGmap' of the gradient map of the adjacent decoded area I'.
  • the above formula (3) and formula (4) are just examples.
  • the embodiments of the present application can also use other methods to determine the gradient map of the adjacent decoded area and the gradient map of the gradient map.
  • the above formulas (3) and (4) are deformed, and the gradient map is determined using the deformed formula, as well as the gradient map of the gradient map.
  • the continuity parameter of the adjacent decoded area is determined based on the gradient map GGmap of the adjacent decoded area.
  • the continuity parameter of the adjacent decoded area is determined based on the gradient map of the adjacent decoded area, and the continuity parameter of the adjacent decoded area is determined as the continuity parameter of the image area.
  • the embodiment of the present application does not limit the method of determining the continuity parameters of adjacent decoded areas based on the gradient map of the gradient map in S602-A1-13 above.
  • the median value of each gradient value in the gradient map of the adjacent decoded area is determined as the continuity parameter of the image area.
  • the average of the gradient values in the gradient maps of adjacent decoded areas is determined as the continuity parameter of the image area.
  • the continuity parameter of the image area is determined by determining the gradient map of the gradient map of the adjacent decoded area.
  • the calculation method is simple and the calculation cost is small, thereby improving the coding efficiency.
  • the method of the following example 2 can also be used to determine the continuity parameter of the image area.
  • Example 2 Determine the continuity parameters of the image area by determining the spatial redundancy of adjacent decoded areas. That is, the above S602-A1 includes the following steps of S602-A1-21 and S602-A1-23:
  • S602-A1-2 Determine adjacent decoded areas of the image area.
  • a decoded area with the same size as the image area among decoded areas adjacent to the image area is regarded as an adjacent decoded area of the image area.
  • the decoded area of the decoded area adjacent to the image area that is inconsistent with the size of the image area is regarded as the adjacent decoded area of the image area.
  • S602-A1-23. Determine the continuity parameter of the image area based on the residual value of the adjacent decoded area, where the residual value is negatively correlated with the continuity parameter.
  • the spatial redundancy of the image is small, it means that the correlation of the image is strong, the changes of pixels in the image are more continuous, and the signal energy of the image is more concentrated. If the spatial redundancy of the image is large, it means that the correlation of the image is weak, the changes of pixels in the image are discontinuous, and the signal energy of the image is not concentrated.
  • the spatial redundancy of an image can be represented by the residual value of the image, that is, a large residual value indicates that the image has large spatial redundancy, and a small residual value indicates that the image has small spatial redundancy.
  • intra-frame prediction is used to calculate the spatial redundancy of the image. Therefore, the embodiment of the present application uses the intra-frame prediction method to calculate the predicted value of the adjacent decoded area, and convert the original value of the adjacent decoded area into The difference from the predicted value is determined as the residual value of the adjacent decoded region.
  • the residual value of an adjacent decoded area may be a superposition of the residual values of each decoded block included in the adjacent decoded area.
  • the adjacent decoded area includes N decoding blocks, N is a positive integer, decode the code stream, and obtain the residual value of each decoding block in the N decoding blocks.
  • the residual values of the N decoding blocks are superimposed to obtain The residual value of this adjacent decoded region.
  • the continuity parameter of the image area is determined based on the residual value of the adjacent decoded area, where the residual value of the adjacent decoded area is negatively correlated with the continuity parameter, that is, the greater the residual value of the adjacent decoded area, the greater the residual value of the adjacent decoded area. , the smaller the continuity parameter of the image area is and the smaller the residual value of the adjacent decoded area is, the larger the continuity parameter of the image area is.
  • the embodiment of the present application does not limit the method of determining the continuity parameters of the image area based on the residual values of adjacent decoded areas.
  • the reciprocal of the residual value of the adjacent decoded area is determined as the continuity parameter of the image area. For example, if the residual value of the adjacent decoded area is a, then the image area is determined The continuity parameter is 1/a.
  • the difference between the preset value and the residual value of the adjacent decoded area is determined as the continuity parameter of the image area.
  • the preset value is a positive number, for example The default value is 1.
  • the continuity parameter of the image area is determined to be 1-a.
  • S602-A2 Determine the first mark according to the continuity parameter of the image area.
  • the first flag is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes, where M is a positive integer greater than 1.
  • the first flag is used to indicate whether the image area is allowed to use the MTS technology, or the first flag is used to indicate the image area. Whether to skip MTS operations.
  • the method of determining the first mark according to the continuity parameter of the image area in the above S602-A2 includes the following examples:
  • Example 1 If the continuity parameter of the image area is less than the preset threshold, the value of the first flag is determined to be the first value, and the first value is used to indicate that the image area is not allowed to determine the transformation method from M candidate transformation methods.
  • Example 2 If the continuity parameter of the image area is greater than or equal to the preset threshold, the value of the first flag is determined to be a second value, and the second value is used to indicate that the image area is allowed to determine the transformation method from M candidate transformation methods.
  • the first value is not equal to the second value.
  • the first value is 0.
  • the second value is 1.
  • the decoder in addition to determining the first flag through the continuity parameter of the image area according to the above-mentioned method one, the decoder can also determine the first flag according to the method two.
  • Method 2 decode the code stream and obtain the first flag.
  • the encoding end determines the first flag based on the continuity parameters of the image area, and writes the first flag into the code stream as shown in Table 3. In this way, the decoding end obtains the first flag by decoding the code stream. logo.
  • the decoding end can quickly obtain the first mark through this second method, which saves computing resources and improves the decoding speed.
  • step S603 is performed to determine the transformation type of the current decoding block based on the first flag.
  • a transform type is also called a transform kernel type.
  • the first flag in the embodiment of this application is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes. In this way, the decoding end can determine the transformation type of the current decoding block based on the first flag.
  • the transformation types include horizontal transformation type trTypeHor and vertical transformation type trTypeVer.
  • Method 1 If the value of the first flag is a first value, the index of the transformation type of the current decoding block is determined to be 0, where the first value is used to indicate that the image area is not allowed to determine the transformation mode from M candidate transformation modes. .
  • the transformation types i.e., transformation methods used in the embodiments of the present application mainly include DCT-II, DCT-VIII, and DST-VII.
  • the index of DCT-II is 0, the index of DCT-VIII is 1, and the index of DST -VII has an index of 2, or, DCT-VIII has an index of 2 and DST-VII has an index of 1.
  • the value of the first flag is a first value (for example, 0)
  • the transformation type includes a horizontal transformation type trTypeHor and a vertical transformation type trTypeVer.
  • the horizontal transformation type trTypeHor and the vertical transformation type of the current decoding block are determined.
  • trTypeVer is both 0, that is, the current decoding block uses DCT-II in both the horizontal and vertical directions.
  • a first flag is added to the standard document as a condition for determining the transformation type.
  • the standard document before modification and the standard document after modification are now compared as follows.
  • IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and lfnst_idx is not equal to 0.
  • cIdx is the color component index
  • 0 represents the brightness component
  • 1 and 2 represent the chrominance component
  • 1 represents the cb component and 2 represents the cr component.
  • IntraSubPartitionsSplitType indicates the division type of the current luminance coding block
  • ISP_NO_SPLIT indicates that the current intra-frame block is not divided.
  • lfnst_idx represents the usage of two inseparable low-frequency transformation cores.
  • IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and lfnst_idx is not equal to 0.
  • the value of the first flag is a first value, such as 0, it is determined that the index of the transform type of the current decoding block is 0, and then the transform matrix is determined based on the transform type index 0.
  • Method 2 if the value of the first flag is a second value, determine the transformation type of the current decoding block according to the second flag, where the second flag is obtained by decoding the code stream, and the second value is used to indicate that the image area is allowed to change from
  • the transformation mode is determined from M candidate transformation modes, and the second flag is used to indicate whether the current picture sequence allows the transformation mode to be determined from the M candidate transformation modes.
  • implicitMtsEnabled is determined based on the second flag, and the transformation type of the current decoding block is determined based on implicitMtsEnabled.
  • the decoder first decodes the code stream to obtain the second flag, and determines based on the second flag that the current picture sequence is allowed to be converted from M
  • the first flag is decoded, and the transformation type of the currently decoded block is determined according to the first flag. Specifically, if the value of the first flag is a first numerical value, the index of the transform type of the current decoding block is determined to be 0. If the value of the first flag is a second numerical value, the index of the transformation type of the current decoding block is determined according to the second flag. Transformation type.
  • the second flag and the first flag are the same flag.
  • the second value indicates that the current image sequence allows the transformation mode to be determined from M candidate transformation modes.
  • the current transformation mode is determined according to the following method. Transform type of decoding block.
  • implicitMtsEnabled is determined to be 1. Otherwise, implicitMtsEnabled is determined to be 0.
  • sps_mts_enabled_flag is used to represent the second flag.
  • the second value is 1.
  • the export process of the above variable implicitMtsEnabled is as follows:
  • IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT
  • cu_sbt_flag is equal to 1 and Max(nTbW,nTbH) is less than or equal to 32
  • sps_explicit_mts_intra_enabled_flag is equal to 0 and CuPredMode[0][xTbY][yTbY] is equal to MODE_INTRA and lfnst_idx is equal to 0 and IntraMipFlag[x0][y0] is equal to 0.
  • cu_sbt_flag indicates whether sub-block transformation is used for the current coding unit.
  • sps_explicit_mts_intra_enabled_flag indicates whether mts_idx will explicitly appear in the entropy coding syntax unit of intra-frame coding.
  • mts_idx is the index specifying the transform core.
  • CuPredMode[0][xTbY][yTbY] represents the prediction mode of the coding unit whose luminance component is at the (xTbY, yTbY) position.
  • MODE_INTRA indicates that the prediction mode is intra prediction
  • the transformation type of the current decoding block is determined according to the implicitMtsEnabled.
  • the transformation type of the current decoding block can be determined in the following manner:
  • cu_sbt_horizontal_flag indicates whether the current coding unit should be divided horizontally into two transformation units
  • cu_sbt_pos_flag indicates whether the tu_y_coded_flag, tu_cb_coded_flag and tu_cr_coded_flag of the first transformation unit of the current coding unit exist.
  • tu_y_coded_flag indicates whether the current luma block contains non-0 transform coefficients
  • tu_cb_coded_flag indicates whether the current chroma block (cb component) contains non-0 transform coefficients
  • tu_cr_coded_flag indicates whether the current chroma block (cr component) contains non-0 transform coefficients.
  • trTypeHor 2
  • trTypeVer 1.
  • implicitMtsEnabled if implicitMtsEnabled is not equal to 1, look up table 5 according to mts_idx to obtain trTypeHor and trTypeVer.
  • mts_idx can be obtained by decoding the code stream.
  • the transformation coefficients of the current decoding block are inversely transformed according to trTypeHor and trTypeVer to obtain the residual value of the current decoding block.
  • the transformation matrix is determined according to the transformation type of the current decoding block and the size of the current decoding block; the transformation matrix is used to perform an inverse transformation operation on the transformation coefficient to obtain the residual value of the current decoding block.
  • the transformation matrix transMatrix is used to inversely transform the transformation coefficient of the current decoding block to obtain the residual value of the current decoding block.
  • the residual value of the current decoding block is obtained:
  • T is the transformation matrix
  • the transformation coefficient can be inversely transformed based on the transformation matrix to obtain the residual value of the current decoding block.
  • the current block is predicted according to the prediction mode of the current decoding block to obtain the prediction value of the current decoding block.
  • the predicted value and the residual value are added to obtain the reconstructed value of the current decoded block.
  • the decoding method in the embodiment of the present application determines the transform coefficient of the current decoding block by decoding the code stream; determines the first flag of the image area where the current encoding block is located, and the first flag is determined based on the continuity parameter of the image area.
  • the first flag Used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes.
  • the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area; according to the first flag, determine the transformation type of the current decoding block; according to the current
  • the transformation type of the decoding block inversely transforms the transformation coefficient of the current decoding block to obtain the residual value of the current decoding block. That is, the embodiment of the present application adds a condition for determining the transformation type of the current decoding block. For example, if the value of the first flag is 0, it is determined that the transformation type of the current decoding block is 0, thereby improving the speed of determining the transformation type. , improve decoding efficiency.
  • Figure 7 is a schematic flowchart of an image decoding method provided by an embodiment of the present application.
  • Figure 7 can be understood as a specific implementation of the decoding method shown in Figure 6.
  • the decoding method in this embodiment of the present application includes:
  • the image area is the current image frame as an example.
  • the embodiment of this application takes MTS technology as an example.
  • the above-mentioned first flag is used to indicate whether the current image frame is allowed to perform an MTS operation.
  • the encoding end writes the first flag into the code stream, so that after the decoding end obtains the code stream, it decodes the code stream to obtain the first flag.
  • the decoding end decodes the code stream and can directly obtain the transform coefficient of the current decoding block from the code stream.
  • the encoding when encoding, the encoding end quantizes the transform coefficients to obtain quantized coefficients, and then encodes the quantized coefficients to obtain a code stream.
  • the decoder decodes the code stream to obtain the quantized coefficients of the current decoded block, and performs inverse quantization on the quantized coefficients to obtain the transform coefficients of the current decoded block.
  • S707 Obtain the reconstruction value of the current decoding block based on the prediction value and residual value of the current decoding block.
  • the decoding method in the embodiment of the present application obtains the first identifier of the current image frame by decoding the code stream. According to the first identifier, the transformation type of the current decoding block is determined; according to the transformation type of the current decoding block, the transformation of the current decoding block is The coefficients are inversely transformed to obtain the residual value of the current decoded block. That is, the embodiment of the present application adds a condition for determining the transformation type of the current decoding block. For example, if the value of the first flag is 0, it is determined that the transformation type of the current decoding block is 0, thereby improving the speed of determining the transformation type. , improve decoding efficiency.
  • the size of the sequence numbers of the above-mentioned processes does not mean the order of execution.
  • the execution order of each process should be determined by its functions and internal logic, and should not be used in this application.
  • the implementation of the examples does not constitute any limitations.
  • the term "and/or" is only an association relationship describing associated objects, indicating that three relationships can exist. Specifically, A and/or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
  • the character "/" in this article generally indicates that the related objects are an "or" relationship.
  • Figure 8 is a schematic block diagram of an image encoding device provided by an embodiment of the present application.
  • the image encoding device 10 includes:
  • Parameter determination unit 11 used to determine the continuity parameter of the image area where the current coding block is located, the continuity parameter being used to indicate the continuity of changes in the pixel points in the image area;
  • Mode determination unit 12 configured to determine the transformation mode of the current coding block according to the continuity parameter of the image area
  • the transformation unit 13 is configured to transform the current coding block according to the transformation method to obtain the transformation coefficient of the current coding block.
  • the parameter determination unit 11 is specifically configured to determine the gradient map of the image area and the gradient map of the gradient map; determine the continuity parameters of the image area based on the gradient map of the gradient map. .
  • the parameter determination unit 11 is specifically configured to determine the average value of the gradient values in the gradient map of the gradient map as the continuity parameter of the image area.
  • the parameter determination unit 11 is specifically configured to determine the residual value of the image area; determine the continuity parameter of the image area according to the residual value of the image area, wherein the residual value Negatively related to the continuity parameter.
  • the mode determination unit 12 is specifically configured to determine the transformation mode of the current coding block to be the first transformation mode if the continuity parameter of the image area is less than a preset threshold.
  • the mode determination unit 12 determines the transformation mode of the current coding block from M candidate transformation modes if the continuity parameter of the image area is greater than or equal to the preset threshold, and the M is greater than or equal to A positive integer of 1.
  • the mode determination unit 12 determines a first flag according to the continuity parameter of the image area, and the first flag is used to indicate whether the image area is allowed to determine a transformation mode from M candidate transformation modes.
  • the M is a positive integer greater than 1; according to the first flag, the transformation mode of the current coding block is determined.
  • the mode determination unit 12 determines that the value of the first flag is a first value if the continuity parameter of the image area is less than a preset threshold, and the first value is used to indicate that it is not allowed.
  • the image area determines a transformation mode from the M candidate transformation modes.
  • the mode determining unit 12 determines that the transformation mode of the current coding block is the first transformation mode if the value of the first flag is the first numerical value.
  • the mode determination unit 12 determines that the value of the first flag is a second value if the continuity parameter of the image area is greater than or equal to the preset threshold, and the second value is The instruction allows the image region to determine a transformation mode from the M candidate transformation modes.
  • the mode determining unit 12 determines the transformation mode of the current coding block from M candidate transformation modes if the value of the first flag is the second value.
  • the mode determination unit 12 determines the rate distortion cost when using the M candidate transformation modes to transform the current coding block; according to the rate distortion costs corresponding to the M candidate transformation modes, from the Select one candidate transformation mode from the M candidate transformation modes as the transformation mode of the current coding block.
  • the mode determination unit 12 determines the candidate transformation mode with the smallest rate distortion cost among the M candidate transformation modes as the transformation mode of the current coding block.
  • the transformation unit 13 is used to write the first flag into the code stream.
  • the image area is the current image frame, or the current image sequence, or the current image slice, or the current coding block.
  • the code stream further includes a second flag, and the third The second flag is used to indicate whether the current picture sequence allows the transformation mode to be determined from the M candidate transformation modes.
  • the first mark multiplexes the second mark.
  • the first conversion mode is DTC-II mode.
  • the M candidate transformation modes include a transformation mode composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode.
  • the M candidate transformation modes include multiple transformation modes corresponding to the multi-transform core selection MTS.
  • the device 10 shown in Figure 8 can execute the encoding method of the embodiment of the present application, and perform the aforementioned and other operations and/or functions of each unit in the device 10 in order to implement the corresponding processes in each method such as the above encoding method.
  • the device 10 shown in Figure 8 can execute the encoding method of the embodiment of the present application, and perform the aforementioned and other operations and/or functions of each unit in the device 10 in order to implement the corresponding processes in each method such as the above encoding method.
  • the device 10 shown in Figure 8 can execute the encoding method of the embodiment of the present application, and perform the aforementioned and other operations and/or functions of each unit in the device 10 in order to implement the corresponding processes in each method such as the above encoding method.
  • Figure 9 is a schematic block diagram of an image decoding device provided by an embodiment of the present application.
  • the image decoding device 20 includes:
  • Decoding unit 21 used to decode the code stream and determine the transform coefficient of the current decoding block
  • the flag determination unit 22 is used to determine the first flag of the image area where the current encoding block is located.
  • the first flag is determined based on the continuity parameter of the image area.
  • the first flag is used to indicate whether the The image area determines the transformation mode from M candidate transformation modes, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;
  • Type determination unit 23 configured to determine the transformation type of the current decoding block according to the first flag
  • the inverse transform unit 24 is configured to inversely transform the transform coefficient of the current decoded block according to the transform type of the current decoded block to obtain the residual value of the current decoded block.
  • the flag determining unit 22 is specifically configured to determine the continuity parameter of the image area; determine the first flag according to the continuity parameter of the image area.
  • the flag determining unit 22 is specifically configured to determine the adjacent decoded area of the image area; determine the gradient map of the adjacent decoded area, and the gradient map of the gradient map; according to the A gradient map of a gradient map that determines the continuity parameters of the image region.
  • the flag determining unit 22 is specifically configured to determine the average value of the gradient values in the gradient map of the gradient map as the continuity parameter of the image area.
  • the flag determining unit 22 is specifically configured to determine adjacent decoded areas of the image area; determine the residual value of the adjacent decoded area; and determine the residual value of the adjacent decoded area according to the residual value of the adjacent decoded area. value to determine the continuity parameter of the image region, wherein the residual value is negatively correlated with the continuity parameter.
  • the flag determination unit 22 is specifically configured to determine that the value of the first flag is a first value if the continuity parameter of the image area is less than a preset threshold, and the first value is used for Indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes.
  • the flag determination unit 22 is specifically configured to determine that the value of the first flag is a second value if the continuity parameter of the image area is greater than or equal to a preset threshold, and the second value Used to indicate that the image area is allowed to determine the transformation mode from the M candidate transformation modes.
  • the flag determining unit 22 is specifically configured to decode the code stream to obtain the first flag.
  • the type determination unit 23 is configured to determine that the index of the transform type of the current decoding block is 0 if the value of the first flag is a first value, and the first value is used for Indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes.
  • the type determination unit 23 is configured to determine the transformation type of the current decoding block according to the second flag if the value of the first flag is a second value, and the second The flag is obtained by decoding the code stream.
  • the second value is used to indicate that the image area is allowed to determine the transformation mode from the M candidate transformation modes.
  • the second flag is used to indicate whether the current picture sequence is allowed to be transformed from all the candidate transformation modes.
  • the transformation method is determined among the M candidate transformation methods.
  • the transformation type includes a horizontal transformation type and a vertical transformation type.
  • the image area is the current image frame, or the current image sequence, or the current image slice, or the current coding block.
  • the second mark and the first mark are the same mark.
  • the inverse transformation unit 24 is configured to determine a transformation matrix according to the transformation type of the current decoding block and the size of the current decoding block; use the transformation matrix to perform an inverse transformation operation on the transformation coefficient, Obtain the residual value of the current decoding block.
  • the M candidate transformation modes include a transformation mode composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode.
  • the M candidate transformation modes include multiple transformation modes corresponding to the multi-transform core selection MTS.
  • the device embodiments and the method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be repeated here.
  • the device 20 shown in FIG. 9 may correspond to the corresponding subject in performing the decoding method of the embodiment of the present application, and the foregoing and other operations and/or functions of each unit in the device 20 are respectively to implement the decoding method and other methods. The corresponding process in , for the sake of brevity, will not be repeated here.
  • the software unit may be located in a mature storage medium in this field such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in combination with its hardware.
  • Figure 10 is a schematic block diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device 30 may be the video encoder or video decoder described in the embodiment of the present application.
  • the electronic device 30 may include:
  • Memory 33 and processor 32 the memory 33 is used to store the computer program 34 and transmit the program code 34 to the processor 32.
  • the processor 32 can call and run the computer program 34 from the memory 33 to implement the method in the embodiment of the present application.
  • the processor 32 may be configured to perform the steps in the above method 200 according to instructions in the computer program 34 .
  • the processor 32 may include but is not limited to:
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • the memory 33 includes but is not limited to:
  • Non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory may be Random Access Memory (RAM), which is used as an external cache.
  • RAM Random Access Memory
  • RAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM double data rate synchronous dynamic random access memory
  • Double Data Rate SDRAM DDR SDRAM
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous link dynamic random access memory
  • Direct Rambus RAM Direct Rambus RAM
  • the computer program 34 can be divided into one or more units, and the one or more units are stored in the memory 33 and executed by the processor 32 to complete the tasks provided by this application.
  • the one or more units may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the execution process of the computer program 34 in the electronic device 30 .
  • the electronic device 30 may also include:
  • Transceiver 33 the transceiver 33 can be connected to the processor 32 or the memory 33 .
  • the processor 32 can control the transceiver 33 to communicate with other devices. Specifically, it can send information or data to other devices, or receive information or data sent by other devices.
  • Transceiver 33 may include a transmitter and a receiver.
  • the transceiver 33 may further include an antenna, and the number of antennas may be one or more.
  • bus system where in addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus.
  • Figure 11 is a schematic block diagram of a video encoding and decoding system provided by an embodiment of the present application.
  • the video encoding and decoding system 40 may include: a video encoder 41 and a video decoder 42, where the video encoder 41 is used to perform the video encoding method involved in the embodiment of the present application, and the video decoder 42 is used to perform
  • the embodiment of the present application relates to a video decoding method.
  • This application also provides a computer storage medium on which a computer program is stored.
  • the computer program When the computer program is executed by a computer, the computer can perform the method of the above method embodiment.
  • embodiments of the present application also provide a computer program product containing instructions, which when executed by a computer causes the computer to perform the method of the above method embodiments.
  • This application also provides a code stream, which is generated by the above encoding method.
  • the code stream includes a first flag.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted over a wired connection from a website, computer, server, or data center (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, server or data center.
  • the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the available media may be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as digital video discs (DVD)), or semiconductor media (such as solid state disks (SSD)), etc.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or may be Integrated into another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separate.
  • a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente demande concerne des procédés et des appareils de codage et de décodage d'image, un dispositif, un système, et un support de stockage. Le procédé consiste à : déterminer un paramètre de continuité d'une région d'image où un bloc de codage courant est situé, le paramètre de continuité étant utilisé pour indiquer une continuité de changement de points de pixel dans la région d'image; déterminer un mode de transformation du bloc de codage courant selon le paramètre de continuité de la région d'image; et transformer le bloc de codage courant selon le mode de transformation pour obtenir un coefficient de transformation du bloc de codage courant. Selon des modes de réalisation de la présente demande, le mode de transformation du bloc de codage courant est déterminé en fonction de la valeur du paramètre de continuité, par exemple, si la valeur du paramètre de continuité est faible, de sorte qu'il est indiqué que l'énergie de signal de la région d'image n'est pas concentrée, un premier mode de transformation par défaut peut être utilisé en tant que mode de transformation du bloc de codage courant, et une opération de sélection du mode de transformation optimal parmi une pluralité de modes de transformation est omise, de sorte qu'un surdébit de calcul inutile soit économisé, et que l'efficacité de codage soit améliorée.
PCT/CN2022/080706 2022-03-14 2022-03-14 Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage WO2023173255A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/080706 WO2023173255A1 (fr) 2022-03-14 2022-03-14 Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/080706 WO2023173255A1 (fr) 2022-03-14 2022-03-14 Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage

Publications (1)

Publication Number Publication Date
WO2023173255A1 true WO2023173255A1 (fr) 2023-09-21

Family

ID=88022026

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/080706 WO2023173255A1 (fr) 2022-03-14 2022-03-14 Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage

Country Status (1)

Country Link
WO (1) WO2023173255A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200404272A1 (en) * 2018-06-08 2020-12-24 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
WO2021096173A1 (fr) * 2019-11-11 2021-05-20 엘지전자 주식회사 Procédé de codage d'image basé sur une conversion, et dispositif associé
CN113574880A (zh) * 2019-03-13 2021-10-29 北京字节跳动网络技术有限公司 关于子块变换模式的分割

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200404272A1 (en) * 2018-06-08 2020-12-24 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
CN113574880A (zh) * 2019-03-13 2021-10-29 北京字节跳动网络技术有限公司 关于子块变换模式的分割
WO2021096173A1 (fr) * 2019-11-11 2021-05-20 엘지전자 주식회사 Procédé de codage d'image basé sur une conversion, et dispositif associé

Similar Documents

Publication Publication Date Title
WO2020119814A1 (fr) Procédé et dispositif de reconstruction d'image
CN114885159B (zh) 位置相关预测组合的模式相关和大小相关块级限制的方法和装置
KR20210107131A (ko) 이미지 예측 방법, 장치 및 시스템, 디바이스 및 저장 매체
US20230319267A1 (en) Video coding method and video decoder
WO2023044868A1 (fr) Procédé de codage vidéo, procédé de décodage vidéo, dispositif, système et support de stockage
WO2023173255A1 (fr) Procédés et appareils de codage et de décodage d'image, dispositif, système et support de stockage
WO2023044919A1 (fr) Procédé, dispositif, et système de codage et de décodage vidéo, et support de stockage
WO2023197229A1 (fr) Procédé, appareil, dispositif et système de codage/décodage vidéo et support de stockage
WO2023220970A1 (fr) Procédé et appareil de codage vidéo, et dispositif, système et support de stockage
WO2023220969A1 (fr) Procédé et appareil de codage vidéo, procédé et appareil de décodage vidéo, et dispositif, système et support d'enregistrement
WO2023236113A1 (fr) Procédés, appareils et dispositifs de codage et de décodage vidéo, système et support de stockage
WO2023184747A1 (fr) Procédé de codage vidéo, procédé de décodage vidéo, appareil, dispositif, système et support de stockage
WO2022116054A1 (fr) Procédé et système de traitement d'images, codeur vidéo et décodeur vidéo
WO2022155922A1 (fr) Procédé et système de codage vidéo, procédé et système de décodage vidéo, codeur vidéo et décodeur vidéo
WO2023220946A1 (fr) Procédé et appareil de codage vidéo, procédé et appareil de décodage vidéo, et dispositif, système et support de stockage
WO2022179394A1 (fr) Procédé de détermination d'échantillon de prédiction de bloc d'image, et dispositifs de codage et de décodage
CN116760976B (zh) 仿射预测决策方法、装置、设备及存储介质
WO2022174475A1 (fr) Procédé et système de codage vidéo, procédé et système de décodage vidéo, codeur vidéo et décodeur vidéo
WO2023221599A1 (fr) Procédé et appareil de filtrage d'image, et dispositif
WO2023122968A1 (fr) Procédé, dispositif et système de prédiction intratrame, et support d'enregistrement
WO2023122969A1 (fr) Procédé de prédiction intra-trame, dispositif, système et support de stockage
WO2023184248A1 (fr) Procédé, appareil, dispositif et système de codage et de décodage vidéo, et support de stockage
WO2024050723A1 (fr) Procédé et appareil de prédiction d'image, et support d'enregistrement lisible par ordinateur
WO2022193390A1 (fr) Procédé et système de codage et de décodage vidéo, et codeur vidéo et décodeur vidéo
WO2022217447A1 (fr) Procédé et système de codage et de décodage vidéo, et codec vidéo

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22931295

Country of ref document: EP

Kind code of ref document: A1