WO2023173255A1

WO2023173255A1 - Image encoding and decoding methods and apparatuses, device, system, and storage medium

Info

Publication number: WO2023173255A1
Application number: PCT/CN2022/080706
Authority: WO
Inventors: 唐桐
Original assignee: Oppo广东移动通信有限公司
Priority date: 2022-03-14
Filing date: 2022-03-14
Publication date: 2023-09-21

Abstract

The present application provides image encoding and decoding methods and apparatuses, a device, a system, and a storage medium. The method comprises: determining a continuity parameter of an image region where a current encoding block is located, the continuity parameter being used for indicating change continuity of pixel points in the image region; determining a transformation mode of the current encoding block according to the continuity parameter of the image region; and transforming the current encoding block according to the transformation mode to obtain a transformation coefficient of the current encoding block. According to embodiments of the present application, the transformation mode of the current encoding block is determined according to the value of the continuity parameter, for example, if the value of the continuity parameter is small, such that it is indicated that signal energy of the image region is not concentrated, a default first transformation mode can be used as a transformation mode of the current encoding block, and an operation of selecting the optimal transformation mode from a plurality of transformation modes is skipped, so that unnecessary calculation overhead is saved, and encoding efficiency is improved.

Description

Image coding and decoding methods, devices, equipment, systems, and storage media

Technical field

The present application relates to the field of video coding and decoding technology, and in particular, to an image coding and decoding method, device, equipment, system, and storage medium.

Background technique

Digital video technology can be incorporated into a variety of video devices, such as digital televisions, smartphones, computers, e-readers, or video players. With the development of video technology, video data includes a larger amount of data. In order to facilitate the transmission of video data, video devices implement video compression technology to make the video data more efficiently transmitted or stored.

Video compression is achieved through coding, and the coding process includes prediction, transformation, and quantization. For example, determine the prediction block of the current block through intra-frame prediction and/or inter-frame prediction, subtract the prediction block from the current block to obtain the residual block, transform the residual block to obtain the transform coefficient, and quantize the transform coefficient to obtain the quantized coefficient, And the quantized coefficients are encoded to form a code stream.

In order to improve the transformation effect, Multiple Transform Selection (MTS) technology is proposed, which selects the optimal transformation method for the current block from multiple transformation methods. However, MTS technology requires multiple calculations of the rate-distortion cost when selecting a transformation method, which results in high computational overhead and low coding efficiency.

Contents of the invention

Embodiments of the present application provide an image encoding and decoding method, device, equipment, system, and storage medium, which determine the transformation method of the current encoding block through the continuity parameters of the image area, avoid unnecessary calculation overhead, and improve the encoding speed.

Determine the continuity parameter of the image area where the current coding block is located, the continuity parameter being used to indicate the continuity of changes in pixels in the image area;

Determine the transformation mode of the current coding block according to the continuity parameter of the image area;

According to the transformation method, the current coding block is transformed to obtain the transformation coefficient of the current coding block.

In a second aspect, embodiments of the present application provide an image decoding method, including:

Decode the code stream and determine the transform coefficient of the current decoding block;

Determine the first flag of the image area where the current coding block is located, the first flag is determined based on the continuity parameter of the image area, and the first flag is used to indicate whether the image area is allowed to transform from M candidates The transformation mode is determined in the mode, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;

Determine the transform type of the current decoding block according to the first flag;

According to the transformation type of the current decoding block, the transformation coefficient of the current decoding block is inversely transformed to obtain the residual value of the current decoding block.

In a third aspect, the present application provides an image encoding device for performing the method in the above first aspect or its respective implementations. Specifically, the encoder includes a functional unit for performing the method in the above-mentioned first aspect or its respective implementations.

In a fourth aspect, the present application provides an image decoding device for performing the method in the above second aspect or its respective implementations. Specifically, the decoder includes a functional unit for performing the method in the above-mentioned second aspect or its respective implementations.

In a fifth aspect, a video encoder is provided, including a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the above first aspect or its respective implementations.

A sixth aspect provides a video decoder, including a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the above second aspect or its respective implementations.

A seventh aspect provides a video encoding and decoding system, including a video encoder and a video decoder. The video encoder is used to perform the method in the above-mentioned first aspect or its various implementations, and the video decoder is used to perform the method in the above-mentioned second aspect or its various implementations.

An eighth aspect provides a chip for implementing any one of the above-mentioned first to second aspects or the method in each implementation manner thereof. Specifically, the chip includes: a processor, configured to call and run a computer program from a memory, so that the device installed with the chip executes any one of the above-mentioned first to second aspects or implementations thereof. method.

A ninth aspect provides a computer-readable storage medium for storing a computer program that causes a computer to execute any one of the above-mentioned first to second aspects or the method in each implementation thereof.

In a tenth aspect, a computer program product is provided, including computer program instructions, which enable a computer to execute any one of the above-mentioned first to second aspects or the methods in each implementation thereof.

An eleventh aspect provides a computer program that, when run on a computer, causes the computer to execute any one of the above-mentioned first to second aspects or the method in each implementation thereof.

A twelfth aspect provides a code stream, which is generated by any one of the above-mentioned first aspects or implementations thereof.

Based on the above technical solution, by determining the continuity parameter of the image area where the current coding block is located, the continuity parameter is used to indicate the continuity of changes in the pixels in the image area; according to the continuity parameter of the image area, the transformation method of the current coding block is determined ; According to the transformation method, transform the current coding block to obtain the transformation coefficient of the current coding block. That is, in this embodiment of the present application, the transformation method of the current coding block is determined according to the size of the continuity parameter. For example, if the continuity parameter is small, indicating that the signal energy in the image area is not concentrated, the default first transformation method can be used as The transformation method of the current coding block, and skips the operation of selecting the optimal transformation method from multiple transformation methods to save unnecessary computing overhead and improve coding efficiency.

Description of the drawings

Figure 1 is a schematic block diagram of a video encoding and decoding system related to an embodiment of the present application;

Figure 2 is a schematic block diagram of a video encoder involved in an embodiment of the present application;

Figure 3 is a schematic block diagram of a video decoder involved in an embodiment of the present application;

Figure 4 is a schematic flow chart of an image encoding method provided by an embodiment of the present application;

Figure 5 is a schematic flow chart of an image encoding method provided by an embodiment of the present application;

Figure 6 is a schematic flow chart of a video decoding method provided by an embodiment of the present application;

Figure 7 is a schematic flow chart of an image decoding method provided by an embodiment of the present application;

Figure 8 is a schematic block diagram of an image encoding device provided by an embodiment of the present application;

Figure 9 is a schematic block diagram of an image decoding device provided by an embodiment of the present application;

Figure 10 is a schematic block diagram of an electronic device provided by an embodiment of the present application;

Figure 11 is a schematic block diagram of a video encoding and decoding system provided by an embodiment of the present application.

Detailed ways

This application can be applied to the fields of image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, real-time video encoding and decoding, etc. For example, the solution of this application can be combined with the audio and video coding standard (AVS for short), such as H.264/audio video coding (AVC for short) standard, H.265/high-efficiency video coding (AVS for short) high efficiency video coding (HEVC) standard and H.266/versatile video coding (VVC) standard. Alternatively, the solution of this application can be operated in conjunction with other proprietary or industry standards, including ITU-TH.261, ISO/IECMPEG-1Visual, ITU-TH.262 or ISO/IECMPEG-2Visual, ITU-TH.263 , ISO/IECMPEG-4Visual, ITU-TH.264 (also known as ISO/IECMPEG-4AVC), including scalable video codec (SVC) and multi-view video codec (MVC) extensions. It should be understood that the technology of this application is not limited to any specific codec standard or technology.

For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced with reference to FIG. 1 .

Figure 1 is a schematic block diagram of a video encoding and decoding system related to an embodiment of the present application. It should be noted that Figure 1 is only an example, and the video encoding and decoding system in the embodiment of the present application includes but is not limited to what is shown in Figure 1 . As shown in FIG. 1 , the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120 . The encoding device is used to encode the video data (which can be understood as compression) to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

The encoding device 110 in the embodiment of the present application can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function. That is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120. Examples include smartphones, desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, vehicle-mounted computers, and the like.

In some embodiments, the encoding device 110 may transmit the encoded video data (eg, code stream) to the decoding device 120 via the channel 130 . Channel 130 may include one or more media and/or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120 .

In one example, channel 130 includes one or more communication media that enables encoding device 110 to transmit encoded video data directly to decoding device 120 in real time. In this example, encoding device 110 may modulate the encoded video data according to the communication standard and transmit the modulated video data to decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum. Optionally, the communication media may also include wired communication media, such as one or more physical transmission lines.

In another example, channel 130 includes a storage medium that can store video data encoded by encoding device 110 . Storage media include a variety of local access data storage media, such as optical disks, DVDs, flash memories, etc. In this example, the decoding device 120 may obtain the encoded video data from the storage medium.

In another example, channel 130 may include a storage server that may store video data encoded by encoding device 110 . In this example, the decoding device 120 may download the stored encoded video data from the storage server. Optionally, the storage server may store the encoded video data and may transmit the encoded video data to the decoding device 120, such as a web server (eg, for a website), a File Transfer Protocol (FTP) server, etc.

In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. Among other things, the output interface 113 may include a modulator/demodulator (modem) and/or a transmitter.

In some embodiments, the encoding device 110 may include a video source 111 in addition to the video encoder 112 and the input interface 113 .

Video source 111 may include at least one of a video capture device (eg, a video camera), a video archive, a video input interface for receiving video data from a video content provider, a computer graphics system Used to generate video data.

The video encoder 112 encodes the video data from the video source 111 to generate a code stream. Video data may include one or more images (pictures) or sequence of pictures (sequence of pictures). The code stream contains the encoding information of an image or image sequence in the form of a bit stream. Encoded information may include encoded image data and associated data. The associated data may include sequence parameter set (SPS), picture parameter set (PPS) and other syntax structures. An SPS can contain parameters that apply to one or more sequences. A PPS can contain parameters that apply to one or more images. A syntax structure refers to a collection of zero or more syntax elements arranged in a specified order in a code stream.

The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113 . The encoded video data can also be stored on a storage medium or storage server for subsequent reading by the decoding device 120 .

In some embodiments, decoding device 120 includes input interface 121 and video decoder 122.

In some embodiments, in addition to the input interface 121 and the video decoder 122, the decoding device 120 may also include a display device 123.

The input interface 121 includes a receiver and/or a modem. Input interface 121 may receive encoded video data over channel 130.

The video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

The display device 123 displays the decoded video data. Display device 123 may be integrated with decoding device 120 or external to decoding device 120 . Display device 123 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

In addition, Figure 1 is only an example, and the technical solutions of the embodiments of the present application are not limited to Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

The video coding framework involved in the embodiments of this application is introduced below.

Figure 2 is a schematic block diagram of a video encoder related to an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression of images (lossy compression), or can also be used to perform lossless compression (lossless compression) of images. The lossless compression can be visually lossless compression (visually lossless compression) or mathematically lossless compression (mathematically lossless compression).

The video encoder 200 can be applied to image data in a luminance-chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2 or 4:4:4, Y represents brightness (Luma), Cb(U) represents blue chroma, Cr(V) represents red chroma, U and V represent Chroma, which is used to describe color and saturation. For example, in the color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 Chroma component (YYYYCbCrCbCr), 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).

For example, the video encoder 200 reads video data, and for each frame of image in the video data, divides one frame of image into several coding tree units (coding tree units, CTU). In some examples, CTB may be called "Tree block", "Largest Coding unit" (LCU for short) or "coding tree block" (CTB for short). Each CTU can be associated with an equal-sized block of pixels within the image. Each pixel can correspond to one luminance (luminance or luma) sample and two chrominance (chrominance or chroma) samples. Therefore, each CTU can be associated with one block of luma samples and two blocks of chroma samples. A CTU size is, for example, 128×128, 64×64, 32×32, etc. A CTU can be further divided into several coding units (Coding Units, CUs) for encoding. CUs can be rectangular blocks or square blocks. CU can be further divided into prediction unit (PU for short) and transform unit (TU for short), thus enabling coding, prediction, and transformation to be separated and processing more flexible. In an example, the CTU is divided into CUs in a quad-tree manner, and the CU is divided into TUs and PUs in a quad-tree manner.

Video encoders and video decoders can support various PU sizes. Assuming that the size of a specific CU is 2N×2N, the video encoder and video decoder can support a PU size of 2N×2N or N×N for intra prediction, and support 2N×2N, 2N×N, N×2N, N×N or similar sized symmetric PU for inter prediction. The video encoder and video decoder can also support 2N×nU, 2N×nD, nL×2N and nR×2N asymmetric PUs for inter prediction.

In some embodiments, as shown in Figure 2, the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform/quantization unit 230, an inverse transform/quantization unit 240, a reconstruction unit 250, and a loop filter unit. 260. Decode the image cache 270 and the entropy encoding unit 280. It should be noted that the video encoder 200 may include more, less, or different functional components.

Optionally, in this application, the current block (current block) may be called the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be called a predicted image block or an image prediction block, and the reconstructed image block may also be called a reconstruction block or an image reconstructed image block.

In some embodiments, prediction unit 210 includes inter prediction unit 211 and intra estimation unit 212. Since there is a strong correlation between adjacent pixels in a video frame, the intra-frame prediction method is used in video encoding and decoding technology to eliminate the spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in the video, the interframe prediction method is used in video coding and decoding technology to eliminate the temporal redundancy between adjacent frames, thereby improving coding efficiency.

The inter-frame prediction unit 211 can be used for inter-frame prediction. Inter-frame prediction can include motion estimation (motion estimation) and motion compensation (motion compensation). It can refer to image information of different frames. Inter-frame prediction uses motion information to find a reference from a reference frame. block, a prediction block is generated based on the reference block to eliminate temporal redundancy; the frames used in inter-frame prediction can be P frames and/or B frames, P frames refer to forward prediction frames, and B frames refer to bidirectional predictions frame. Inter-frame prediction uses motion information to find reference blocks from reference frames and generate prediction blocks based on the reference blocks. The motion information includes the reference frame list where the reference frame is located, the reference frame index, and the motion vector. The motion vector can be in whole pixels or sub-pixels. If the motion vector is in sub-pixels, then interpolation filtering needs to be used in the reference frame to make the required sub-pixel blocks. Here, the reference frame found according to the motion vector is A block of whole pixels or sub-pixels is called a reference block. Some technologies will directly use the reference block as a prediction block, and some technologies will process the reference block to generate a prediction block. Reprocessing to generate a prediction block based on a reference block can also be understood as using the reference block as a prediction block and then processing to generate a new prediction block based on the prediction block.

The intra-frame estimation unit 212 only refers to the information of the same frame image and predicts the pixel information in the current coded image block to eliminate spatial redundancy. The frames used in intra prediction may be I frames.

Intra-frame prediction has multiple prediction modes. Taking the international digital video coding standard H series as an example, the H.264/AVC standard has 8 angle prediction modes and 1 non-angle prediction mode, and H.265/HEVC has been extended to 33 angles. prediction mode and 2 non-angle prediction modes. The intra-frame prediction modes used by HEVC include planar mode (Planar), DC and 33 angle modes, for a total of 35 prediction modes. The intra-frame modes used by VVC include Planar, DC and 65 angle modes, for a total of 67 prediction modes.

It should be noted that with the increase of angle modes, intra-frame prediction will be more accurate and more in line with the development needs of high-definition and ultra-high-definition digital videos.

Residual unit 220 may generate a residual block of the CU based on the pixel block of the CU and the prediction block of the PU. For example, residual unit 220 may generate a residual block of a CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the PU of the CU. Predict the corresponding sample in the block.

Transform/quantization unit 230 may quantize the transform coefficients. Transform/quantization unit 230 may quantize transform coefficients associated with the TU of the CU based on quantization parameter (QP) values associated with the CU. Video encoder 200 may adjust the degree of quantization applied to transform coefficients associated with the CU by adjusting the QP value associated with the CU.

Inverse transform/quantization unit 240 may apply inverse quantization and inverse transform to the quantized transform coefficients, respectively, to reconstruct the residual block from the quantized transform coefficients.

Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to produce a reconstructed image block associated with the TU. By reconstructing blocks of samples for each TU of a CU in this manner, video encoder 200 can reconstruct blocks of pixels of the CU.

The loop filtering unit 260 is used to process the inversely transformed and inversely quantized pixels to compensate for distortion information and provide a better reference for subsequent encoding of pixels. For example, a deblocking filtering operation can be performed to reduce the number of pixel blocks associated with the CU. block effect.

In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive compensation/adaptive loop filtering (SAO/ALF) unit, where the deblocking filtering unit is used to remove blocking effects, and the SAO/ALF unit Used to remove ringing effects.

Decoded image cache 270 may store reconstructed pixel blocks. Inter prediction unit 211 may perform inter prediction on PUs of other images using reference images containing reconstructed pixel blocks. Additionally, intra estimation unit 212 may use the reconstructed pixel blocks in decoded image cache 270 to perform intra prediction on other PUs in the same image as the CU.

Entropy encoding unit 280 may receive the quantized transform coefficients from transform/quantization unit 230 . Entropy encoding unit 280 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy encoded data.

Figure 3 is a schematic block diagram of a video decoder related to an embodiment of the present application.

As shown in FIG. 3 , the video decoder 300 includes an entropy decoding unit 310 , a prediction unit 320 , an inverse quantization/transformation unit 330 , a reconstruction unit 340 , a loop filtering unit 350 and a decoded image cache 360 . It should be noted that the video decoder 300 may include more, less, or different functional components.

Video decoder 300 can receive the code stream. Entropy decoding unit 310 may parse the codestream to extract syntax elements from the codestream. As part of parsing the code stream, the entropy decoding unit 310 may parse entropy-encoded syntax elements in the code stream. The prediction unit 320, the inverse quantization/transformation unit 330, the reconstruction unit 340 and the loop filtering unit 350 may decode the video data according to the syntax elements extracted from the code stream, that is, generate decoded video data.

In some embodiments, prediction unit 320 includes inter prediction unit 321 and intra estimation unit 322.

Intra estimation unit 322 may perform intra prediction to generate predicted blocks for the PU. Intra estimation unit 322 may use an intra prediction mode to generate predicted blocks for a PU based on pixel blocks of spatially neighboring PUs. Intra estimation unit 322 may also determine the intra prediction mode of the PU based on one or more syntax elements parsed from the codestream.

The inter prediction unit 321 may construct a first reference image list (List 0) and a second reference image list (List 1) according to syntax elements parsed from the code stream. Additionally, if the PU uses inter-prediction encoding, entropy decoding unit 310 may parse the motion information of the PU. Inter prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. Inter prediction unit 321 may generate a predictive block for the PU based on one or more reference blocks of the PU.

Inverse quantization/transform unit 330 may inversely quantize (ie, dequantize) transform coefficients associated with a TU. Inverse quantization/transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.

After inversely quantizing the transform coefficients, inverse quantization/transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to produce a residual block associated with the TU.

Reconstruction unit 340 uses the residual blocks associated with the TU of the CU and the prediction blocks of the PU of the CU to reconstruct the pixel blocks of the CU. For example, reconstruction unit 340 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

Loop filtering unit 350 may perform deblocking filtering operations to reduce blocking artifacts for blocks of pixels associated with the CU.

Video decoder 300 may store the reconstructed image of the CU in decoded image cache 360 . The video decoder 300 may use the reconstructed image in the decoded image cache 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

The basic process of video encoding and decoding is as follows: at the encoding end, an image frame is divided into blocks. For the current block, the prediction unit 210 uses intra prediction or inter prediction to generate a prediction block of the current block. The residual unit 220 may calculate a residual block based on the prediction block and the original block of the current block, that is, the difference between the prediction block and the original block of the current block. The residual block may also be called residual information. The residual block undergoes transformation and quantization processes such as transformation/quantization unit 230 to remove information that is insensitive to human eyes to eliminate visual redundancy. Optionally, the residual block before transformation and quantization by the transformation/quantization unit 230 may be called a time domain residual block, and the time domain residual block after transformation and quantization by the transformation/quantization unit 230 may be called a frequency residual block. or frequency domain residual block. The entropy encoding unit 280 receives the quantized change coefficient output from the change quantization unit 230, and may perform entropy encoding on the quantized change coefficient to output a code stream. For example, the entropy encoding unit 280 may eliminate character redundancy according to the target context model and probability information of the binary code stream.

At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 320 uses intra prediction or inter prediction for the current block based on the prediction information to generate a prediction block of the current block. The inverse quantization/transform unit 330 uses the quantization coefficient matrix obtained from the code stream to perform inverse quantization and inverse transformation on the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstruction block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or based on the blocks to obtain a decoded image. The encoding end also needs similar operations as the decoding end to obtain the decoded image. The decoded image may also be called a reconstructed image, and the reconstructed image may be used as a reference frame for inter-frame prediction for subsequent frames.

It should be noted that the block division information determined by the encoding end, as well as mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., are carried in the code stream when necessary. The decoding end determines the same block division information as the encoding end by parsing the code stream and analyzing the existing information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information, thereby ensuring the decoded image obtained by the encoding end It is the same as the decoded image obtained by the decoding end.

The current block can be the current coding unit (CU) or the current prediction unit (PU), etc.

The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the block-based hybrid coding framework. The basic process of the video codec, but is not limited to this framework and process.

It can be seen from the above that when encoding, the general hybrid coding framework will first perform prediction, and the prediction uses the correlation performance in space or time to obtain an image that is the same or similar to the current block. For a block, it is possible that the predicted block and the current block are exactly the same, but it is difficult to guarantee that this is true for all blocks in a video, especially for natural videos, or videos captured by cameras, because there are The presence of noise. Moreover, irregular motion, distortion, occlusion, brightness, etc. changes in the video are difficult to completely predict. Therefore, the hybrid coding framework will subtract the predicted image from the original image of the current block to obtain the residual image, or the current block minus the predicted block will obtain the residual block. Residual blocks are usually much simpler than the original image, so prediction can significantly improve compression efficiency. The residual block is not directly encoded, but is usually transformed first. Transformation is to transform the residual image from the spatial domain to the frequency domain and remove the correlation of the residual image. After the residual image is transformed into the frequency domain, since the energy is mostly concentrated in the low-frequency region, most of the transformed non-zero coefficients are concentrated in the upper left corner. Next, quantization is used for further compression. And since the human eye is not sensitive to high frequencies, a larger quantization step size can be used in high-frequency areas.

After the original image undergoes Discrete Cosine Transform (DCT), there are non-zero coefficients in the upper left corner area.

The current common transformation method is DCT-II. However, for some image contents, DCT-II cannot achieve optimal transformation. Therefore, multiple transformation kernels are designed to adaptively select the optimal transformation kernel for different image blocks, which is closer to the theoretical optimum. Excellent transformation performance.

Multiple Transform Selection (MTS) technology is a newly added transformation technology in VVC. In addition to the DCT-II transform in HEVC, VVC introduces two new transform methods, discrete sine transform DST-VII and DCT-VIII. Intra-frame and inter-frame coding blocks can choose from these three transform methods. Table 1 below shows the transform Basic functions of the method:

Table 1

Among them, in the above Table 1, N is the number of points of the original signal, ω ₀ is the compensation coefficient,

Since the images are all 2-dimensional, the amount of computation and memory overhead of directly performing two-dimensional transformation were unacceptable to the hardware conditions at the time. Therefore, when using the above-mentioned DCT-II, DCT-VIII, and DST-VII in the standard for transformation, , both are split into horizontal and vertical directions, and a two-step one-dimensional transformation is performed. For example, perform horizontal transformation first and then vertical transformation, or perform vertical transformation first and then horizontal transformation.

In order to control the complexity of MTS, whether to use MTS is marked at the image sequence level (SPS level) within the frame and between frames. If the SPS level mark indicates that MTS can be used, each CU also has a Flag to mark whether the current CU uses it. MTS. The flag of MTS at the CU level will only exist when the following two conditions are met:

1) Both length and width are less than or equal to 32;

2) The current TU is in non-transform skip mode;

It should be pointed out that MTS only applies to the brightness component.

When CU-level MTS exists, the transformation process is roughly divided into three steps:

(1) Use DCT-II transformation on the current CU to calculate its rate distortion cost RD1.

(2) Use a new transformation for the current CU. The transformation method is selected from the candidate set composed of the combination of DST-VII and DCT-VIII. The candidate set is shown in Table 2 below, and the rate distortion costs RD2, RD3... are calculated.

Table 2

(3) Select the transformation that minimizes rate distortion cost as the final transformation method of the current CU.

An implementation of an encoder using MTS is as follows. First, the input image is divided into multiple non-overlapping CTU blocks. Then, each CTU is processed in sequence according to the raster scanning order, and the CTU is divided into several CUs according to the prediction cost. The general steps to determine the optimal transformation method for each CU are as follows: (1) If the MTS flag of the current CU=0, then Directly select DCT-II as the transformation method of the current CU; (2) Otherwise, first use DCT-II transformation on the current CU, calculate its rate distortion cost RDCostTmp1, and then select the candidate transformation method in Table 2 to transform the current CU, select The transformation method with the smallest rate distortion cost is MTOpt, and the rate distortion cost is RDCostTmp2. (3) Compare RDCostTmp1 and RDCostTmp2, and select the transformation with less cost as the transformation method of the current CU. Finally, the residual block is transformed, quantized, and entropy encoded according to the optimal block division method, and the prediction information such as the block division mode is encoded, and the output code stream is waiting for transmission.

An implementation of a decoder using MTS is as follows. First, the input code stream undergoes entropy decoding, inverse quantization, and inverse transformation. The inverse transformation process mainly includes the following steps: (1) First, derive the transformation type trType based on code stream information such as sps_mts_enabled_flag. (2) Derive the transformation matrix transMatrix according to the transformation type trType and CU size. (3) Use the transformation matrix transMatrix and the transformation coefficient to calculate the inverse transformation value to obtain the prediction residual value, and reconstruct the image based on the residual block. Finally, the reconstructed image is sent to the DBF/SAO/ALF filter, and the filtered image is sent to the buffer to wait for video playback.

It can be seen from the above that although MTS technology can select the optimal transformation kernel for an image block, it requires multiple calculations of the rate-distortion cost when selecting the transformation kernel, so it requires a large amount of calculation overhead.

On the other hand, the essence of MTS is to select the deformation body of DCT (DCT-VIII, DST-VII) for transformation. The premise to ensure the effectiveness of DCT is that the signal energy of the image block is concentrated, so that the high-frequency components can be transformed after DCT transformation. Concentrated in the upper left corner, most areas in the lower right corner are low-frequency components. In other words, only for images with concentrated signal energy, DCT transformation can achieve better compression performance. On the contrary, when the signal energy is not concentrated, the DCT transformation performance is poor, so the performance gain brought by choosing the best among the poor is very limited.

Therefore, when encoding video images whose signal energy is not concentrated, MTS technology calculations are likely to generate a lot of unnecessary overhead, waste computing resources, and increase encoding time.

In order to solve the above technical problem, the embodiment of the present application first determines the continuity parameter of the image area where the current coding block is located. The continuity parameter is used to indicate the continuity of changes in pixels in the image area, where the continuity of changes in pixels can also be Understood as the correlation of pixels. Due to the poor continuity of the image area, it means that the signal energy is not concentrated, and the DCT transformation effect is poor. At this time, there is no need to spend a lot of computing resources to select the optimal transformation method from multiple transformation methods. Based on this, the embodiment of the present application determines the transformation method of the current coding block according to the continuity parameter of the image area. For example, if the continuity parameter of the image area is small, indicating that the signal energy of the image area is not concentrated, DTC- II method for transformation, there is no need to select the optimal transformation method from multiple transformation methods for transformation, thus avoiding unnecessary calculation overhead and improving coding speed.

The image encoding and decoding method provided by the embodiment of the present application will be introduced below with reference to specific embodiments.

First, with reference to Figure 4, the video encoding method provided by the embodiment of the present application is introduced, taking the encoding end as an example.

FIG. 4 is a schematic flowchart of an image encoding method provided by an embodiment of the present application. The embodiment of the present application is applied to the video encoder shown in FIGS. 1 and 2 .

As shown in Figure 4, the method in the embodiment of this application includes:

S401. Determine the continuity parameter of the image area where the current coding block is located.

This continuity parameter is used to indicate the continuity of changes in pixels in the image area.

During the image encoding process, the encoder receives a video stream, which is composed of a series of image frames, and performs video encoding on each frame of image in the video stream. The video encoder divides the image frames into blocks to obtain the current encoding block.

In some embodiments, the current coding block is also called a current block, a current image block, a coding block, a current coding unit, a current block to be coded, a current image block to be coded, etc.

When dividing blocks, the blocks divided by the traditional method contain both the chrominance component of the current coding block position and the luminance component of the current coding block position. The separation tree technology (dual tree) can be divided into separate component blocks, such as separate brightness blocks and separate chroma blocks. The brightness block can be understood as containing only the brightness component of the current coding block position, and the chroma block can be understood as containing only the current coding block position. Encode the chroma component of the block position. In this way, luminance components and chrominance components at the same position can belong to different blocks, and the division can be more flexible. If the separation tree is used in CU partitioning, then some CUs contain both luminance and chrominance components, some CUs only contain luminance components, and some CUs only contain chrominance components.

In some embodiments, the current coding block in the embodiment of the present application only includes chrominance components, which can be understood as chrominance blocks.

In some embodiments, the current coding block in the embodiment of the present application only includes the luminance component, which can be understood as a luminance block.

In some embodiments, the current coding block includes both luma and chrominance components.

The encoding process in this embodiment of the present application is to divide the current image frame into blocks to obtain the current coding block, and use the intra-frame or inter-frame prediction method to predict the current coding block to obtain the prediction value of the current coding block. The original value of the current coding block is subtracted from the predicted value to obtain the residual value of the current coding block. Determine the transformation method corresponding to the current coding block, use this transformation method to transform the residual value, and obtain the transformation coefficient. Optionally, the transform coefficients are quantized, and the quantized transform coefficients are encoded to obtain a code stream.

The embodiment of the present application involves the transformation process in the above encoding process.

It can be seen from the above description that adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods can improve the transformation effect. However, for images where the signal energy is not concentrated, the compression effect brought by the transformation is poor. At this time, adaptively selecting the optimal transformation method for the current coding block from multiple transformation methods to transform these images whose signal energy is not concentrated not only has an insignificant compression effect, but also generates unnecessary computational overhead and increases encoding time.

Therefore, in this embodiment of the present application, before transforming the current coding block, it first determines whether the signal energy in the image area where the current coding block is located is concentrated. For an image, if the changes in pixel points in the image are discontinuous, it means that the spatial correlation of the image is weak and the signal energy is not concentrated. If the changes in pixels in the image are continuous, it means that the spatial correlation of the image is strong and the signal energy is concentrated.

Based on this, the embodiment of the present application uses the size of the continuity parameter of the image area to indicate whether the signal energy of the image area is concentrated. For example, if the continuity parameter of the image area is large, it indicates the intensity of the pixels in the image area. The change is continuous, which indicates that the signal energy in this image area is concentrated. If the continuity parameter of the image area is small, it indicates that the changes in pixel points in the image area are discontinuous, which further indicates that the signal energy in the image area is not concentrated. For image areas where signal energy is not concentrated, selecting the optimal transformation method from multiple transformation methods will not only fail to bring significant transformation effects, but also generate unnecessary computational overhead and increase encoding time. Therefore, in the embodiment of the present application, For image areas where signal energy is not concentrated, skip the operation of selecting the optimal transformation method from multiple transformation methods (such as skipping MTS technology) to save unnecessary computing overhead and improve coding efficiency.

In this way, before transformation, this embodiment of the present application first determines the continuity parameter of the image area where the current coding block is located. The continuity parameter is used to indicate the continuity of changes in pixels in the image area. According to the size of the continuity parameter, the transformation method of the current coding block is determined. For example, if the continuity parameter is small, the default first transformation method is used as the transformation method of the current coding block, and the selection of the best transformation method from multiple transformation methods is skipped. Optimize transformation operations to save unnecessary computing overhead and improve coding efficiency.

The embodiment of the present application does not limit the size of the image area, and it can be any image area including the current coding block.

In some embodiments, the image area is the current image frame, and the continuity parameter of the image area is used to indicate the continuity of changes in pixel points in the current image frame. In this way, the embodiment of the present application can determine the transformation method of any coding block in the current image frame according to the continuity parameter of the current image frame.

In some embodiments, the image area is the current image sequence, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image sequence. In this way, the embodiment of the present application can determine the transformation method of any coding block in the current image sequence based on the continuity parameter of the current image sequence.

In some embodiments, the image area is the current image patch, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image patch. In this way, the embodiment of the present application can determine the transformation method of any coding block in the current image slice according to the continuity parameter of the current image slice.

In some embodiments, the image area is the current coding block, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current coding block. In this way, when encoding each coding block in this embodiment of the present application, it is necessary to determine the continuity parameters of each coding block to determine the transformation method of each coding block.

In the embodiment of this application, the methods for determining the continuity parameters of the image area include but are not limited to the following:

Method 1 is to determine the continuity parameters of the image area through the gradient map, that is, the above S401 includes the following steps of S401-A1 and S401-A2:

S401-A1. Determine the gradient map of the image area and the gradient map of the gradient map;

S401-A2: Determine the continuity parameter of the image area according to the gradient map of the gradient map.

The gradient map of the image area can reflect the change trend of the pixels in the image area, and the gradient map of the gradient map of the image area can reflect the change rate of the pixels in the image area, that is, the continuity of change.

Therefore, in the first method, the continuity parameter of the image area is determined by calculating the gradient map of the gradient map of the image area.

For example, the gradient map Gmap of the image area is determined according to the following formula (1):

Gmap(x,y)＝|I(x,y)-I(x+1,y)|+|I(x,y)-I(x,y+1)| (1)

Among them, I(x, y) is the pixel value of image area I at position (x, y), I(x+1, y) is the pixel value of image area I at position (x+1, y), I(x, y+1) is the pixel value of image area I at position (x, y+1), Gmap(x, y) is the gradient value of the pixel point at position (x, y) in image area I .

For example, the gradient map GGmap of the gradient map of the image area is determined according to the following formula (2):

GGmap(x,y)＝|Gmap(x,y)-Gmap(x+1,y)|+|Gmap(x,y)-Gmap(x,y+1)| (2)

Among them, Gmap(x+1,y) is the gradient value of the pixel at position (x+1,y) in image area I, Gmap(x,y+1) is the position (x,y+ The gradient value of the pixel at 1), where Gmap(x+1,y) and Gmap(x,y+1) can be determined according to the above formula (1). GGmap(x,y) is the gradient value of the gradient value of the pixel at position (x,y) in image area I.

The above formula (1) and formula (2) show that the gradient value and the gradient value of the pixel point (x, y) of the image area I are determined. Other pixel points in the image area I refer to the pixel point (x, y). ) method, determine the gradient value of each pixel in the image area I, and obtain the gradient map Gmap of the image area I. Use the above formula (2) to calculate the gradient value corresponding to each pixel in the gradient map Gmap of the image area I, and then obtain the gradient map GGmap of the gradient map of the image area I.

It should be noted that the above formula (1) and formula (2) are just examples. The embodiments of the present application can also use other methods to determine the gradient map of the image area and the gradient map of the gradient map. For example, the above formula ( 1) and formula (2) are deformed, and the gradient map is determined using the deformed formula, as well as the gradient map of the gradient map.

According to the above method, after the gradient map of the gradient map of the image area is determined, the continuity parameter of the image area is determined based on the gradient map GGmap of the gradient map.

The embodiment of the present application does not limit the method of determining the continuity parameter of the image area based on the gradient map of the gradient map in S401-A2 above.

In one example, the median value of each gradient value in the gradient map of the image region is determined as the continuity parameter of the image region.

In another example, the average value of the gradient values in the gradient map of the image area is determined as the continuity parameter of the image area.

The above-mentioned method 1 determines the continuity parameter of the image area by determining the gradient map of the gradient map of the image area. The calculation method is simple and the calculation cost is small, thereby improving the coding efficiency.

In some embodiments, in addition to using the above method one to determine the continuity parameter of the image area, the following method two can also be used to determine the continuity parameter of the image area.

The second method is to determine the continuity parameters of the image area by determining the spatial redundancy of the image area. That is, the above S401 includes the following steps of S401-B1 and S401-B2:

S401-B1. Determine the residual value of the image area;

S401-B2. Determine the continuity parameter of the image area according to the residual value of the image area, where the residual value is negatively correlated with the continuity parameter.

If the spatial redundancy of the image area is small, it means that the correlation of the image area is strong, the changes of pixel points in the image area are more continuous, and the signal energy of the image area is more concentrated. If the spatial redundancy of the image area is large, it means that the correlation of the image area is weak, the changes of pixel points in the image area are discontinuous, and the signal energy of the image area is not concentrated. The spatial redundancy of the image area can be represented by the residual value of the image area, that is, a large residual value indicates that the spatial redundancy of the image area is large, and a small residual value indicates that the spatial redundancy of the image area is small.

In the image coding process, intra-frame prediction is used to calculate the spatial redundancy of the image. Therefore, the embodiment of the present application uses the intra-frame prediction method to calculate the predicted value of the image area, and calculates the difference between the original value and the predicted value of the image area. Determine the residual value of this image area.

During encoding, encoding is performed in units of blocks. Therefore, in the embodiment of the present application, the residual value of the image area may be a superposition of the residual values of each coding block included in the image area. For example, the image area includes N coding blocks, and N is a positive integer. Intra-frame prediction is used to determine the predicted value of each coding block. For each coding block, the original value of the coding block is subtracted from the predicted value, and we get The residual value of this coding block. The residual values of N coding blocks are superimposed to obtain the residual value of the image area.

Next, according to the residual value of the image area, the continuity parameter of the image area is determined. The residual value of the image area is negatively correlated with the continuity parameter. That is, the greater the residual value of the image area, the greater the continuity parameter of the image area. The smaller the value, the smaller the residual value of the image area, and the greater the continuity parameter of the image area.

The embodiment of the present application does not limit the method of determining the continuity parameter of the image area based on the residual value of the image area.

In a possible implementation, the reciprocal of the residual value of the image area is determined as the continuity parameter of the image area. For example, if the residual value of the image area is a, then the continuity parameter of the image area is determined to be 1. /a.

In another possible implementation, the difference between the preset value and the residual value of the image area is determined as the continuity parameter of the image area. Optionally, the preset value is a positive number, such as the preset value. The value is 1. For example, if the residual value of the image area is a, then the continuity parameter of the image area is determined to be 1-a.

Optionally, other methods can be used to determine the continuity parameter of the image area based on the residual value of the image area. The embodiment of the present application does not limit this, as long as it is ensured that the residual value of the image area is negatively correlated with the continuity parameter. That’s it.

After determining the continuity parameter of the image area according to the above-mentioned method 1 or 2, the following step S402 is performed to determine the transformation method of the current coding block based on the continuity parameter of the image area.

S402. Determine the transformation method of the current coding block according to the continuity parameter of the image area.

In the embodiment of the present application, the continuity parameter of the image area is used to indicate the continuity of changes in the pixels in the image area. If the continuity parameter of the image area is small, it means that the signal energy in the image area is not concentrated. In this case, DTC is used The transformation cannot bring significant compression effect. In this case, calculating the rate distortion cost of multiple transformation methods to select the optimal transformation method from these multiple transformation methods will not only not bring beneficial compression effects, but also The computational cost is high. Therefore, when the continuity parameter of the image area is large, the operation of selecting the optimal transformation method from multiple transformation methods is skipped to save computing resources and improve coding efficiency.

In the above S402, the methods for determining the transformation method of the current coding block according to the continuity parameter of the image area include but are not limited to the following:

Method 1: If the continuity parameter of the image area is less than the preset threshold, it is determined that the transformation mode of the current coding block is the first transformation mode.

It can be seen from the above description that if the continuity parameter of the image area is less than the preset threshold, it means that the signal energy of the image area is not concentrated. At this time, if the optimal transformation method is selected from multiple transformation methods (for example, using MTS technology) , not only cannot bring beneficial compression effects, but also increases unnecessary calculations. Therefore, in this case, the encoder skips the operation of selecting the optimal transform scheme from multiple transform schemes, for example, skips the MTS operation, and directly determines the first transform scheme as the target transform scheme of the current coding block.

In a possible implementation, the above-mentioned first transformation mode is one of multiple preset transformation modes. For example, the first transformation mode is DTC-II mode, DST-VII mode and DCT-VIII mode. any transformation method.

Optionally, the first conversion mode is the DTC-II mode.

The embodiment of the present application does not place a limit on the preset threshold, which is determined based on actual needs.

Method 2: If the continuity parameter of the image area is greater than or equal to the preset threshold, the transformation method of the current coding block is determined from M candidate transformation methods, where M is a positive integer greater than 1.

If the continuity parameter of the image area is greater than or equal to the preset threshold, it means that the signal energy in the image area is relatively concentrated. At this time, selecting the optimal transformation method from multiple transformation methods (such as M transformation methods) can improve Compression effect. Therefore, in this case, the encoder performs an operation of selecting the optimal transform method from M transform methods, such as performing an MTS operation, to determine the transform method of the current coding block from these M candidate transform methods.

This embodiment of the present application does not limit the number and types of M candidate transformation methods.

In some embodiments, the M candidate transformation modes include transformation modes composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode. Since the images are all 2-dimensional, the amount of computation and memory overhead of directly performing two-dimensional transformation were unacceptable to the hardware conditions at the time. Therefore, when using the above-mentioned DCT-II, DCT-VIII, and DST-VII in the standard for transformation, , both are split into horizontal and vertical directions, and a two-step one-dimensional transformation is performed. For example, perform horizontal transformation first and then vertical transformation, or perform vertical transformation first and then horizontal transformation. Based on this, the M candidate transformation modes in the embodiment of the present application can be any combination of DTC-II mode, DST-VII mode and DCT-VIII mode in the horizontal and vertical directions. For example, candidate transformation method 1 is DTC-II in the horizontal direction and DST-VII in the vertical direction; candidate transformation method 2 is DTC-II in the horizontal direction and DCT-VIII in the vertical direction; candidate transformation method 3 is The horizontal direction is DST-VII mode and the vertical direction is DTC-Ⅱ mode. The horizontal direction of candidate transformation method 4 is DST-VII mode and the vertical direction is DCT-VIII mode. The horizontal direction of candidate transformation method 5 is DCT- The VIII mode and the vertical direction are DTC-Ⅱ mode. The horizontal direction and vertical direction of candidate transformation mode 6 are DCT-VIII mode and the vertical direction are DST-VII mode. The horizontal and vertical modes of candidate transformation mode 7 are both DTC-Ⅱ mode, the horizontal direction and the vertical mode of candidate transformation mode 8 are both DST-VII mode, and the horizontal direction and vertical mode of candidate transformation mode 9 are both DCT-VIII mode.

In some embodiments, the M candidate transformation modes include multiple transformation modes corresponding to MTS, such as the DTC-II mode, and the combined transformation modes shown in Table 2 above.

In the second method, determining the transformation mode of the current coding block from M candidate transformation modes includes the following steps:

Step 1: Determine the rate distortion cost when using M candidate transformation methods to transform the current coding block;

Step 2: Select a candidate transformation method from the M candidate transformation methods as the transformation method of the current coding block according to the rate distortion cost corresponding to the M candidate transformation methods.

In a possible implementation, the method of determining the transformation mode of the current coding block from the M candidate transformation modes may be: first, use the first candidate transformation mode among the M candidate transformation modes to transform the current coding block. , and calculate the rate distortion cost corresponding to the first candidate transformation method. If the rate distortion cost corresponding to the first candidate transformation method is less than or equal to the preset value, the first candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the first candidate transformation method is greater than the preset value, use the second candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the corresponding rate distortion cost of the second candidate transformation method. rate distortion cost. If the rate distortion cost corresponding to the second candidate transformation method is less than or equal to the preset value, the second candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the second candidate transformation method is greater than the preset value, repeat the above steps, use the third candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the third candidate transformation method The rate distortion cost corresponding to the transformation method. By analogy, a candidate transformation method is selected from M candidate transformation methods as the transformation method of the current coding block.

In another possible implementation, the method of determining the transformation mode of the current coding block from M candidate transformation modes may be: using these M candidate transformation modes to transform the current coding block respectively, and calculating the M candidates The rate distortion cost corresponding to each candidate transformation method in the transformation method. Among the M candidate transformation modes, the candidate transformation mode with the smallest rate distortion cost is determined as the transformation mode of the current coding block.

In the above-mentioned methods one and two, the encoder determines the transformation method of the current coding block according to the continuity parameter of the image area.

In some embodiments, in addition to using the above-mentioned method 1 and 2 to determine the transformation mode of the current coding block, the encoder may also use the following method 3 to determine the transformation mode of the current coding block.

In the third method, the encoder first determines a flag based on the continuity parameter of the image area, and determines the transformation mode of the current coding block based on the flag. That is, the above S402 includes the following steps of S402-A1 and S402-A2:

S402-A1. Determine the first mark according to the continuity parameter of the image area.

The first flag is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes, where M is a positive integer greater than 1.

In some embodiments, if the above M candidate transformation methods are M candidate transformation methods corresponding to the MTS technology, the first flag is used to indicate whether the image area is allowed to use the MTS technology, or the first flag is used to indicate the image area. Whether to skip MTS operations.

Among them, the method of determining the first mark according to the continuity parameter of the image area in the above S402-A1 includes the following examples:

Example 1: If the continuity parameter of the image area is less than the preset threshold, it means that the signal energy in the image area is not concentrated. At this time, the value of the first flag is determined to be the first value. The first value is used to indicate that the image area is not allowed. The transformation method is determined from M candidate transformation methods.

Example 2, if the continuity parameter of the image area is greater than or equal to the preset threshold, it means that the signal energy in the image area is concentrated. At this time, the value of the first flag is determined to be the second value. The second value is used to indicate that the image area is allowed to change from The transformation method is determined among M candidate transformation methods.

Wherein, the first value is not equal to the second value.

The embodiments of this application do not limit the specific values of the first numerical value and the second numerical value.

Optional, the first value is 0.

Optional, the second value is 1.

S402-A2. Determine the transformation mode of the current coding block according to the first flag.

According to the method of S402-A1, after the first flag of the image area is determined, the transformation mode of the current coding block is determined based on the first flag.

As can be seen from the above, if the value of the first flag is the first value, it indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes. At this time, the step of determining the transformation mode from the M candidate transformation modes is skipped. operation, but directly determines the default first transformation mode as the transformation mode of the current coding block.

If the value of the first flag is the second value, it indicates that the image area is allowed to determine the transformation mode from M candidate transformation modes. At this time, the transformation mode of the current coding block is determined from the M candidate transformation modes.

For determining the transformation mode of the current coding block from the M candidate transformation modes, reference may be made to the description of the second mode above.

For example, the current coding block is transformed using the first candidate transformation method among the M candidate transformation methods, and the rate distortion cost corresponding to the first candidate transformation method is calculated. If the rate distortion cost corresponding to the first candidate transformation method is less than or equal to the preset value, the first candidate transformation method is used as the transformation method of the current coding block. If the rate distortion cost corresponding to the first candidate transformation method is greater than the preset value, use the second candidate transformation method among the M candidate transformation methods to transform the current coding block, and calculate the corresponding rate distortion cost of the second candidate transformation method. rate distortion cost. By analogy, a candidate transformation method is selected from M candidate transformation methods as the transformation method of the current coding block.

For another example, the M candidate transformation methods are used to transform the current coding block respectively, and the rate distortion cost corresponding to each of the M candidate transformation methods is calculated. Among the M candidate transformation modes, the candidate transformation mode with the smallest rate distortion cost is determined as the transformation mode of the current coding block.

In this embodiment of the present application, after the encoder determines the transformation mode of the current coding block according to the above method, it performs the following steps of S403.

S403. Transform the current coding block according to the transformation method to obtain the transformation coefficient of the current coding block.

Specifically, the current coding block is predicted using the intra prediction method and/or the inter prediction method to obtain the prediction value of the current coding block. Then, based on the prediction value of the current coding block and the current coding block, the prediction value of the current coding block is obtained. Residual value. For example, subtract the predicted value from the pixel value of the current coding block to obtain the residual value of the current coding block.

Then, use the transformation method determined by the above method to transform the residual value of the current coding block to obtain the transformation coefficient of the current coding block.

In one example, if the transformation method of the current coding block determined according to the above method is the DCT-II transformation method, then according to the formula shown in Table 2 above, the DCT-II transformation method is used to transform the residual of the current coding block Transform the value to obtain the transformation coefficient of the current coding block.

In another example, if the transformation method of the current coding block determined according to the above method is the DCT-VIII transformation method, then according to the formula shown in Table 2 above, the DCT-VIII transformation method is used to transform the residual of the current coding block. The difference value is transformed to obtain the transformation coefficient of the current coding block.

In another example, if the transformation method of the current coding block determined according to the above method is the DST-VII transformation method, then according to the formula shown in Table 2 above, the DST-VII transformation method is used to transform the residual of the current coding block. The difference value is transformed to obtain the transformation coefficient of the current coding block.

In some embodiments, the encoder writes the above-mentioned first flag into the code stream, so that the decoder determines the transform type based on the first flag and implements inverse transformation of the transform coefficients.

In some embodiments, if the image area is any one of the current image frame, the current image slice, and the current coding block, the code stream also includes a second flag. The second flag is used to indicate whether the current picture sequence is allowed to be copied from The transformation method is determined among M candidate transformation methods. In the MTS technology, the second flag is specifically used to indicate whether the current image sequence allows the use of the MTS technology. If the second flag indicates that the current image sequence is not allowed to use the MTS technology, it is determined that all coded image blocks under the current image sequence do not use the MTS technology, and the first transformation method is determined as the target transformation method to calculate the residual value. Transform to get the transformation coefficients. If the second flag indicates that the current image sequence is allowed to use the MTS technology, then all coded image blocks under the current image sequence use the MTS technology to determine the transformation method.

In some embodiments, if the image area is the current image sequence, the first flag multiplexes the second flag.

The embodiment of the present application does not limit the specific positions of the first mark and the second mark in the code stream.

For example, taking MTS technology as an example, pic_mts_enable_flag can be used to represent the first flag.

For example, sps_explicit_mts_inter_enabled_flag may be used to represent the second flag.

In some embodiments, the first flag and the second flag are located in the coding unit. At this time, part of the syntax of the coding unit is as shown in Table 3:

table 3

In the embodiment of the present application, a constraint first flag is added to the existing coding unit syntax, and the first flag is used to indicate whether the current area skips the MTS operation. In this way, the decoding end can determine the transformation type trType of the current coding block according to the first flag during decoding.

The image coding method provided by the embodiment of the present application determines the continuity parameter of the image area where the current coding block is located. The continuity parameter is used to indicate the continuity of changes in the pixel points in the image area; based on the continuity parameter of the image area, the current coding block is determined based on the continuity parameter of the image area. The transformation method of the coding block; according to the transformation method, the current coding block is transformed to obtain the transformation coefficient of the current coding block. That is, in this embodiment of the present application, the transformation method of the current coding block is determined according to the size of the continuity parameter. For example, if the continuity parameter is small, indicating that the signal energy in the image area is not concentrated, the default first transformation method can be used as The transformation method of the current coding block, and skips the operation of selecting the optimal transformation method from multiple transformation methods to save unnecessary computing overhead and improve coding efficiency.

Figure 5 is a schematic flowchart of an image encoding method provided by an embodiment of the present application. Figure 5 can be understood as a more specific encoding method of the encoding method shown in Figure 4. As shown in Figure 5, the encoding method according to the embodiment of the present application includes the following steps:

S501. Determine the continuity parameter of the current image frame.

In the embodiment of this application, the image area is the current image frame as an example. At this time, the continuity parameter is used to indicate the continuity of changes in pixels in the current image frame.

In some embodiments, a gradient map of the current image frame is determined, and a gradient map of the gradient map is determined, and a continuity parameter of the current image frame is determined based on the gradient map of the gradient map. For example, the average value of the gradient values in the gradient map of the current image frame is determined as the continuity parameter of the current image frame.

In some embodiments, a residual value of the current image frame is determined, and a continuity parameter of the current image frame is determined based on the residual value of the current image frame, where the residual value is negatively correlated with the continuity parameter.

It should be noted that the implementation of the above S501 is basically the same as the implementation of the above S401. For details, refer to the description of the above S401, which will not be described again here.

S502. Determine whether the continuity parameter of the current image frame is less than a preset threshold.

If the continuity parameter of the current image frame is less than the preset threshold, the following steps of S503 are performed. If the continuity parameter of the current image frame is greater than or equal to the preset threshold, the following steps of S504 are performed.

S503. The current image frame skips the MTS operation, and sets the value of the first flag to the first value.

The embodiment of this application takes MTS technology as an example.

If the continuity parameter of the current image frame is less than the preset threshold, it means that the signal energy of the current image frame is not concentrated. In order to save computing resources and improve coding effects, the current image frame skips the MTS operation. That is to say, all coding blocks under the current image frame do not use MTS technology to determine the transformation method.

Optionally, at this time, the default first transformation mode is determined as the transformation mode of all coding blocks in the current image frame.

For example, pic_mts_enable_flag is used to represent the first flag.

Optional, the first value is 0.

At this time, let pic_mts_enable_flag=0.

Optionally, the first transformation mode is the DCT-II mode.

S504. Perform an MTS operation on the current image frame, and set the value of the first flag to the second value.

If the continuity parameter of the current image frame is greater than or equal to the preset threshold, it means that the signal energy of the current image frame is concentrated. In order to improve the transformation effect, the current image frame is allowed to perform MTS operations. At this time, when determining the specific transformation method for the coding block in the current image frame, MTS technology can be used to determine the optimal transformation method.

For example, pic_mts_enable_flag is used to represent the first flag.

Optional, the second value is 1.

At this time, let pic_mts_enable_flag=1.

S505. Determine the transformation mode of the current coding block according to the first flag.

That is, in the embodiment of this application, when encoding the current coding block, if the MTS flag of the current coding block=0 or pic_mts_enable_flag=0, the DCT-II method is directly selected as the transformation method of the current coding block.

If the MTS flag of the current coding block = 1 and pic_mts_enable_flag = 1, first use DCT-II transformation on the current coding block, calculate its rate distortion cost RDCostTmp1, and then select the candidate transformation method in Table 2 to transform the current coding block, and select The transformation method MTOpt with the smallest rate distortion cost, and the rate distortion cost corresponding to this MTOpt is RDCostTmp2. Compare RDCostTmp1 and RDCostTmp2, and select the transformation method with less cost as the transformation method of the current coding block.

S506: Transform the current coding block using the transformation method of the current coding block to obtain transformation coefficients.

For the implementation of the above S506, refer to the description of the above S403, which will not be described again here.

S507. Encode the transform coefficient to obtain a code stream.

In one example, the transform coefficients of the current transform block are directly encoded without quantization to obtain a code stream.

In one example, the transform coefficient of the current encoding block is quantized to obtain the quantized coefficient, and the quantized coefficient is encoded to obtain the code stream.

Further, the encoder writes the above-mentioned first flag into the code stream, so that the decoder determines the transformation type corresponding to the inverse transformation according to the first flag.

The image coding method provided by the embodiment of the present application determines the continuity parameter of the current image frame, and compares the continuity parameter with the preset threshold. If the continuity parameter is less than the preset threshold, the MTS operation is skipped and the MTS operation is skipped. The first transformation mode is determined as the transformation mode of the current coding block, thereby saving unnecessary calculation overhead and improving coding efficiency.

The encoding method of the embodiment of the present application is introduced above. On this basis, the decoding method provided by the embodiment of the present application is introduced below.

FIG. 6 is a schematic flowchart of a video decoding method provided by an embodiment of the present application. The embodiment of the present application is applied to the decoders shown in FIGS. 1 and 3 . As shown in Figure 6, the method in the embodiment of this application includes:

S601. Decode the code stream and determine the transform coefficient of the current decoding block.

In some embodiments, the current decoding block may also be referred to as the current block, the current decoding unit, the decoding block, the block to be decoded, the current block to be decoded, etc.

In some embodiments, when the currently decoded block includes a chroma component and does not include a luma component, the currently decoded block may be called a chroma block.

In some embodiments, when the currently decoded block includes a luma component and does not include a chrominance component, the currently decoded block may be called a luma block.

The above-mentioned transform coefficients are transform coefficients obtained by transforming the residual value of the current decoding block at the encoding end.

In this embodiment, in the above-mentioned S601, the decoding end decodes the code stream and obtains the transform coefficients of the current decoded block, including but not limited to the following:

Method 1: If the encoding end is encoding, the transform coefficients are not quantized, but the transform coefficients are directly encoded to obtain a code stream. In this way, the decoding end decodes the code stream and can directly obtain the transform coefficient of the current decoding block from the code stream.

Method 2: During encoding, the encoding end quantizes the transform coefficients to obtain quantized coefficients, and then encodes the quantized coefficients to obtain a code stream. In this way, the decoder decodes the code stream to obtain the quantized coefficients of the current decoded block, and performs inverse quantization on the quantized coefficients to obtain the transform coefficients of the current decoded block.

S602. Determine the first flag of the image area where the current coding block is located.

The above-mentioned first flag is used to determine the transform type of the current coding block.

The above-mentioned first flag is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes.

The first flag in the embodiment of the present application is determined based on the continuity parameter of the image area. The continuity parameter is used to indicate the continuity of changes in the pixel points in the image area.

Based on this, the embodiment of the present application uses the size of the continuity parameter of the image area to indicate whether the signal energy of the image area is concentrated. For example, if the continuity parameter of the image area is large, it indicates the intensity of the pixels in the image area. The change is continuous, which indicates that the signal energy in this image area is concentrated. If the continuity parameter of the image area is small, it indicates that the changes in pixel points in the image area are discontinuous, which further indicates that the signal energy in the image area is not concentrated. For image areas where signal energy is not concentrated, selecting the optimal transformation method from multiple transformation methods will not only fail to bring significant transformation effects, but also generate unnecessary computational overhead and increase encoding and decoding time. Therefore, in the embodiment of the present application, , for image areas where signal energy is not concentrated, skip the operation of selecting the optimal transformation method from M candidate transformation methods (such as skipping MTS technology) to save unnecessary computing overhead.

The embodiment of the present application does not limit the size of the image area, and it can be any image area including the current decoding block.

In some embodiments, the image area is the current image frame, and the continuity parameter of the image area is used to indicate the continuity of changes in pixel points in the current image frame. Correspondingly, the first flag is used to indicate whether the current image frame is allowed to determine the transformation mode from M candidate transformation modes.

In some embodiments, the image area is the current image sequence, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image sequence. Correspondingly, the first flag is used to indicate whether the current image sequence is allowed to determine the transformation mode from M candidate transformation modes.

In some embodiments, the image area is the current image patch, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current image patch. Correspondingly, the first flag is used to indicate whether the current image slice is allowed to determine the transformation mode from M candidate transformation modes.

In some embodiments, the image area is the current coding block, and the continuity parameter of the image area is used to indicate the continuity of changes in pixels in the current coding block. Correspondingly, the first flag is used to indicate whether the current coding block is allowed to determine the transformation mode from M candidate transformation modes.

In the embodiment of the present application, when inversely transforming the transform coefficients of the current decoding block, the transform type trType needs to be determined first, and then the transform coefficients of the current decoding block are inversely transformed according to the transform type trType to obtain the residual value of the current decoding block. In the embodiment of the present application, the transformation type trType is determined according to the first flag. For example, when the first flag is 0, it means that the image area does not allow the optimal transformation method to be selected from M candidate transformation methods, and then the current decoding block can be determined. The transformation type trType is 0.

In the embodiment of this application, the methods for determining the first mark include but are not limited to the following:

Method 1: The decoder determines the first flag based on the continuity parameter of the image area. Specifically, the above S602 includes the following steps S602-A1 and S602-A2, that is, first determines the continuity parameter of the image area, and then determines the first flag based on the continuity parameter of the image area. Parameters determine the first marker of the image area.

S602-A1. Determine the continuity parameters of the image area.

When the decoding end determines the continuity parameter of the image area, the image area is not completely decoded. At this time, the continuity parameters of adjacent decoded areas of the image area can be used as the continuity parameters of the image area.

In the embodiments of this application, the methods for determining the continuity parameters of the image area include but are not limited to the following examples:

Example 1, determine the continuity parameters of the image area through the gradient map, that is, the above S602-A1 includes the following steps from S602-A1-11 to S602-A1-13:

S602-A1-11. Determine adjacent decoded areas of the image area.

In some embodiments, adjacent decoded regions are consistent in size with image regions.

In some embodiments, adjacent decoded regions are not consistent in size with image regions.

S602-A1-12. Determine the gradient map of the adjacent decoded area and the gradient map of the gradient map.

The gradient map of an image can reflect the change trend of pixels in the image, and the gradient map of the image can reflect the rate of change of pixels in the image, that is, the continuity of change.

Therefore, in the embodiment of the present application, the continuity parameter of the adjacent decoded region can be determined by calculating the gradient map of the gradient map of the adjacent decoded region.

For example, the gradient map Gmap' of the adjacent decoded area is determined according to the following formula (3):

Gmap'(x,y)＝|I'(x,y)-I'(x+1,y)|+|I'(x,y)-I'(x,y+1)| (3)

Among them, I'(x, y) is the pixel value of the adjacent decoded area I' at the position (x, y), I'(x+1, y) is the adjacent decoded area I' at the position (x +1, y), I'(x, y+1) is the pixel value of the adjacent decoded area I' at position (x, y+1), Gmap'(x, y) is the corresponding The gradient value of the pixel at position (x, y) in the adjacent decoded area I'.

For example, the gradient map GGmap' of the gradient map of the adjacent decoded area is determined according to the following formula (4):

GGmap'(x,y)＝|Gmap'(x,y)-Gmap'(x+1,y)|+|Gmap'(x,y)-Gmap'(x,y+1)| (4)

Among them, Gmap'(x+1,y)' is the gradient value of the pixel at position (x+1,y) in the adjacent decoded area I', and Gmap(x,y+1)' is the adjacent decoded area I'. The gradient value of the pixel at position (x, y+1) in area I', where Gmap'(x+1,y) and Gmap'(x,y+1) can be determined according to the above formula (3). GGmap’ (x, y) is the gradient value of the gradient value of the pixel at position (x, y) in the adjacent decoded area I’.

The above formulas (3) and (4) show that the gradient value of the pixel point (x, y) in the adjacent decoded area I' and the gradient value of the gradient value are determined. Other pixels in the adjacent decoded area I' By referring to the pixel point (x, y), the gradient value of each pixel point in the adjacent decoded area I' is determined, and the gradient map Gmap' of the adjacent decoded area I' is obtained. Use the above formula (4) to calculate the gradient value corresponding to each pixel point in the gradient map Gmap' of the adjacent decoded area I', and then obtain the gradient map GGmap' of the gradient map of the adjacent decoded area I'.

It should be noted that the above formula (3) and formula (4) are just examples. The embodiments of the present application can also use other methods to determine the gradient map of the adjacent decoded area and the gradient map of the gradient map. For example, The above formulas (3) and (4) are deformed, and the gradient map is determined using the deformed formula, as well as the gradient map of the gradient map.

According to the above method, after the gradient map of the gradient map of the adjacent decoded area is determined, the continuity parameter of the adjacent decoded area is determined based on the gradient map GGmap of the adjacent decoded area.

S602-A1-13. Determine the continuity parameter of the image area based on the gradient map of the gradient map of the adjacent decoded area.

In this embodiment of the present application, the continuity parameter of the adjacent decoded area is determined based on the gradient map of the adjacent decoded area, and the continuity parameter of the adjacent decoded area is determined as the continuity parameter of the image area.

The embodiment of the present application does not limit the method of determining the continuity parameters of adjacent decoded areas based on the gradient map of the gradient map in S602-A1-13 above.

In one example, the median value of each gradient value in the gradient map of the adjacent decoded area is determined as the continuity parameter of the image area.

In another example, the average of the gradient values in the gradient maps of adjacent decoded areas is determined as the continuity parameter of the image area.

In the above example 1, the continuity parameter of the image area is determined by determining the gradient map of the gradient map of the adjacent decoded area. The calculation method is simple and the calculation cost is small, thereby improving the coding efficiency.

In some embodiments, in addition to using the above example 1 to determine the continuity parameter of the image area, the method of the following example 2 can also be used to determine the continuity parameter of the image area.

Example 2: Determine the continuity parameters of the image area by determining the spatial redundancy of adjacent decoded areas. That is, the above S602-A1 includes the following steps of S602-A1-21 and S602-A1-23:

S602-A1-21. Determine adjacent decoded areas of the image area.

For example, a decoded area with the same size as the image area among decoded areas adjacent to the image area is regarded as an adjacent decoded area of the image area.

For another example, the decoded area of the decoded area adjacent to the image area that is inconsistent with the size of the image area is regarded as the adjacent decoded area of the image area.

S602-A1-22. Determine the residual value of the adjacent decoded area;

S602-A1-23. Determine the continuity parameter of the image area based on the residual value of the adjacent decoded area, where the residual value is negatively correlated with the continuity parameter.

If the spatial redundancy of the image is small, it means that the correlation of the image is strong, the changes of pixels in the image are more continuous, and the signal energy of the image is more concentrated. If the spatial redundancy of the image is large, it means that the correlation of the image is weak, the changes of pixels in the image are discontinuous, and the signal energy of the image is not concentrated. The spatial redundancy of an image can be represented by the residual value of the image, that is, a large residual value indicates that the image has large spatial redundancy, and a small residual value indicates that the image has small spatial redundancy.

During the image decoding process, intra-frame prediction is used to calculate the spatial redundancy of the image. Therefore, the embodiment of the present application uses the intra-frame prediction method to calculate the predicted value of the adjacent decoded area, and convert the original value of the adjacent decoded area into The difference from the predicted value is determined as the residual value of the adjacent decoded region.

During decoding, decoding is performed in units of blocks. Therefore, in this embodiment of the present application, the residual value of an adjacent decoded area may be a superposition of the residual values of each decoded block included in the adjacent decoded area. For example, the adjacent decoded area includes N decoding blocks, N is a positive integer, decode the code stream, and obtain the residual value of each decoding block in the N decoding blocks. The residual values of the N decoding blocks are superimposed to obtain The residual value of this adjacent decoded region.

Next, the continuity parameter of the image area is determined based on the residual value of the adjacent decoded area, where the residual value of the adjacent decoded area is negatively correlated with the continuity parameter, that is, the greater the residual value of the adjacent decoded area, the greater the residual value of the adjacent decoded area. , the smaller the continuity parameter of the image area is and the smaller the residual value of the adjacent decoded area is, the larger the continuity parameter of the image area is.

The embodiment of the present application does not limit the method of determining the continuity parameters of the image area based on the residual values of adjacent decoded areas.

In a possible implementation, the reciprocal of the residual value of the adjacent decoded area is determined as the continuity parameter of the image area. For example, if the residual value of the adjacent decoded area is a, then the image area is determined The continuity parameter is 1/a.

In another possible implementation, the difference between the preset value and the residual value of the adjacent decoded area is determined as the continuity parameter of the image area. Optionally, the preset value is a positive number, for example The default value is 1. For example, if the residual value of an adjacent decoded area is a, then the continuity parameter of the image area is determined to be 1-a.

Optionally, other methods can be used to determine the continuity parameters of the image area based on the residual values of adjacent decoded areas. The embodiments of the present application do not limit this, as long as the residual values of adjacent decoded areas are ensured. Just be negatively correlated with the continuity parameter.

According to the above example 1 or example 2, after the continuity parameter of the image area is determined, the following steps of S602-A2 are performed to determine the first flag according to the continuity parameter of the image area.

S602-A2. Determine the first mark according to the continuity parameter of the image area.

Among them, the method of determining the first mark according to the continuity parameter of the image area in the above S602-A2 includes the following examples:

Example 1: If the continuity parameter of the image area is less than the preset threshold, the value of the first flag is determined to be the first value, and the first value is used to indicate that the image area is not allowed to determine the transformation method from M candidate transformation methods.

Example 2: If the continuity parameter of the image area is greater than or equal to the preset threshold, the value of the first flag is determined to be a second value, and the second value is used to indicate that the image area is allowed to determine the transformation method from M candidate transformation methods.

Wherein, the first value is not equal to the second value.

Optional, the first value is 0.

Optional, the second value is 1.

In the embodiment of the present application, in addition to determining the first flag through the continuity parameter of the image area according to the above-mentioned method one, the decoder can also determine the first flag according to the method two.

Method 2: decode the code stream and obtain the first flag.

In the second method, the encoding end determines the first flag based on the continuity parameters of the image area, and writes the first flag into the code stream as shown in Table 3. In this way, the decoding end obtains the first flag by decoding the code stream. logo.

The decoding end can quickly obtain the first mark through this second method, which saves computing resources and improves the decoding speed.

In this embodiment of the present application, after the decoding end determines the first flag of the image area according to the above-mentioned method 1 or 2, the following step S603 is performed to determine the transformation type of the current decoding block based on the first flag.

S603. Determine the transformation type of the current decoding block according to the first flag.

In some embodiments, a transform type is also called a transform kernel type.

The first flag in the embodiment of this application is used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes. In this way, the decoding end can determine the transformation type of the current decoding block based on the first flag.

Among them, the transformation types include horizontal transformation type trTypeHor and vertical transformation type trTypeVer.

Method 1: If the value of the first flag is a first value, the index of the transformation type of the current decoding block is determined to be 0, where the first value is used to indicate that the image area is not allowed to determine the transformation mode from M candidate transformation modes. .

As can be seen from the above, the transformation types (i.e., transformation methods) used in the embodiments of the present application mainly include DCT-II, DCT-VIII, and DST-VII. Among them, the index of DCT-II is 0, the index of DCT-VIII is 1, and the index of DST -VII has an index of 2, or, DCT-VIII has an index of 2 and DST-VII has an index of 1. When the subsequent transformation matrix is exported, it is exported according to the index of the transformation type. In this embodiment of the present application, if the value of the first flag is a first value (for example, 0), it is determined that the index of the transform type of the current decoding block is 0, that is, it is determined that the transform type of the current decoding block is DCT-II.

In some embodiments, the transformation type includes a horizontal transformation type trTypeHor and a vertical transformation type trTypeVer. When the value of the first flag is a first value (for example, 0), the horizontal transformation type trTypeHor and the vertical transformation type of the current decoding block are determined. trTypeVer is both 0, that is, the current decoding block uses DCT-II in both the horizontal and vertical directions.

In the embodiment of this application, a first flag is added to the standard document as a condition for determining the transformation type. The standard document before modification and the standard document after modification are now compared as follows.

The standard document before modification is:

The horizontal transformation type trTypeHor and the vertical transformation type trTypeVer are exported as follows:

Sets trTypeHor and trTypeVer equal to 0 if one or more of the following conditions are true.

Condition 1, cIdx is greater than 0;

Condition 2, IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and lfnst_idx is not equal to 0.

Among them, cIdx is the color component index, 0 represents the brightness component, 1 and 2 represent the chrominance component, where 1 represents the cb component and 2 represents the cr component. IntraSubPartitionsSplitType indicates the division type of the current luminance coding block, and ISP_NO_SPLIT indicates that the current intra-frame block is not divided. lfnst_idx represents the usage of two inseparable low-frequency transformation cores.

The revised standard document is:

Condition 1, cIdx is greater than 0;

Condition 3: the first flag is equal to the first value, for example, the first flag=0.

It can be seen from the above method 1 that when the value of the first flag is a first value, such as 0, it is determined that the index of the transform type of the current decoding block is 0, and then the transform matrix is determined based on the transform type index 0.

Method 2, if the value of the first flag is a second value, determine the transformation type of the current decoding block according to the second flag, where the second flag is obtained by decoding the code stream, and the second value is used to indicate that the image area is allowed to change from The transformation mode is determined from M candidate transformation modes, and the second flag is used to indicate whether the current picture sequence allows the transformation mode to be determined from the M candidate transformation modes.

Specifically, implicitMtsEnabled is determined based on the second flag, and the transformation type of the current decoding block is determined based on implicitMtsEnabled.

In some embodiments, if the first flag is a picture-level flag and the second flag is a sequence-level flag, the decoder first decodes the code stream to obtain the second flag, and determines based on the second flag that the current picture sequence is allowed to be converted from M When the transformation mode is determined among the candidate transformation modes, the first flag is decoded, and the transformation type of the currently decoded block is determined according to the first flag. Specifically, if the value of the first flag is a first numerical value, the index of the transform type of the current decoding block is determined to be 0. If the value of the first flag is a second numerical value, the index of the transformation type of the current decoding block is determined according to the second flag. Transformation type.

In some embodiments, if the image area is the current image sequence, the second flag and the first flag are the same flag.

In a possible implementation, if the value of the second flag is a second value, the second value indicates that the current image sequence allows the transformation mode to be determined from M candidate transformation modes. At this time, the current transformation mode is determined according to the following method. Transform type of decoding block.

The following takes MTS technology as an example.

If the value of the second flag is the second value and at least one preset condition is true, then implicitMtsEnabled is determined to be 1. Otherwise, implicitMtsEnabled is determined to be 0.

For example, sps_mts_enabled_flag is used to represent the second flag.

Optional, the second value is 1.

In some embodiments, the export process of the above variable implicitMtsEnabled is as follows:

Set implicitMtsEnabled to 1 if sps_mts_enabled_flag equals 1 and one or more of the following conditions are true:

Condition 1, IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT

Condition 2, cu_sbt_flag is equal to 1 and Max(nTbW,nTbH) is less than or equal to 32

Condition 3, sps_explicit_mts_intra_enabled_flag is equal to 0 and CuPredMode[0][xTbY][yTbY] is equal to MODE_INTRA and lfnst_idx is equal to 0 and IntraMipFlag[x0][y0] is equal to 0.

If sps_mts_enabled_flag is equal to 0, or when neither condition 1 nor condition 2 above is true, implicitMtsEnabled is set to 0.

Among them, cu_sbt_flag indicates whether sub-block transformation is used for the current coding unit. sps_explicit_mts_intra_enabled_flag indicates whether mts_idx will explicitly appear in the entropy coding syntax unit of intra-frame coding. mts_idx is the index specifying the transform core. CuPredMode[0][xTbY][yTbY] represents the prediction mode of the coding unit whose luminance component is at the (xTbY, yTbY) position. MODE_INTRA indicates that the prediction mode is intra prediction

After the implicitMtsEnabled is determined according to the above steps, the transformation type of the current decoding block is determined according to the implicitMtsEnabled.

In some embodiments, if implicitMtsEnabled is equal to 1, the transformation type of the current decoding block can be determined in the following manner:

Method 1, if cu_sbt_flag is equal to 1, query trTypeHor and trTypeVer in Table 4 based on cu_sbt_horizontal_flag and cu_sbt_pos_flag.

Table 4

cu_sbt_horizontal_flagcu_sbt_horizontal_flag	cu_sbt_pos_flagcu_sbt_pos_flag	trTypeHortrTypeHor	trTypeVertrTypeVer
00	00	22	11
00	11	11	11
11	00	11	22
11	11	11	11

Among them, cu_sbt_horizontal_flag indicates whether the current coding unit should be divided horizontally into two transformation units, and cu_sbt_pos_flag indicates whether the tu_y_coded_flag, tu_cb_coded_flag and tu_cr_coded_flag of the first transformation unit of the current coding unit exist. Among them, tu_y_coded_flag indicates whether the current luma block contains non-0 transform coefficients, tu_cb_coded_flag indicates whether the current chroma block (cb component) contains non-0 transform coefficients, and tu_cr_coded_flag indicates whether the current chroma block (cr component) contains non-0 transform coefficients.

For example, when cu_sbt_horizontal_flag and cu_sbt_pos_flag are both equal to 0, then trTypeHor is 2 and trTypeVer is 1.

Method 2, if cu_sbt_flag is equal to 0, trTypeHor and trTypeVer are derived according to the following formulas (5) and (6):

trTypeHor=(nTbW>=4&&nTbW<=16)? 1:0 (5)

trTypeVer=(nTbH>=4&&nTbH<=16)? 1:0 (6)

In some embodiments, if implicitMtsEnabled is not equal to 1, look up table 5 according to mts_idx to obtain trTypeHor and trTypeVer.

Among them, mts_idx can be obtained by decoding the code stream.

table 5

mts_idxmts_idx	11	22	33	44
trTypeHortrTypeHor	11	22	11	22
trTypeVertrTypeVer	11	11	22	22

For example, if mts_idx is 1, it is determined that both trTypeHor and trTypeVer are 1.

According to the above steps, after the transformation types trTypeHor and trTypeVer of the current decoding block are determined, the transformation coefficients of the current decoding block are inversely transformed according to trTypeHor and trTypeVer to obtain the residual value of the current decoding block.

S604. According to the transformation type of the current decoding block, perform inverse transformation on the transformation coefficient of the current decoding block to obtain the residual value of the current decoding block.

Specifically, the transformation matrix is determined according to the transformation type of the current decoding block and the size of the current decoding block; the transformation matrix is used to perform an inverse transformation operation on the transformation coefficient to obtain the residual value of the current decoding block.

For example, take the current decoding block size nTbS and the transformation type trType as input, and output the corresponding transformation matrix transMatrix.

Then, the transformation matrix transMatrix is used to inversely transform the transformation coefficient of the current decoding block to obtain the residual value of the current decoding block.

For example, according to the following formula (7), the residual value of the current decoding block is obtained:

in,

is the transformation coefficient of the current block, T is the transformation matrix,

is the residual value of the current decoded block.

According to the above formula (7), the transformation coefficient can be inversely transformed based on the transformation matrix to obtain the residual value of the current decoding block.

It should be noted that the above formula (7) is just an example, and the implementation of the above S604 includes but is not limited to the above formula (7).

In some embodiments, the current block is predicted according to the prediction mode of the current decoding block to obtain the prediction value of the current decoding block.

According to the above steps, after obtaining the residual value and the predicted value of the current decoded block, the predicted value and the residual value are added to obtain the reconstructed value of the current decoded block.

The decoding method in the embodiment of the present application determines the transform coefficient of the current decoding block by decoding the code stream; determines the first flag of the image area where the current encoding block is located, and the first flag is determined based on the continuity parameter of the image area. The first flag Used to indicate whether the image area is allowed to determine the transformation mode from M candidate transformation modes. The continuity parameter is used to indicate the continuity of changes in the pixel points in the image area; according to the first flag, determine the transformation type of the current decoding block; according to the current The transformation type of the decoding block, inversely transforms the transformation coefficient of the current decoding block to obtain the residual value of the current decoding block. That is, the embodiment of the present application adds a condition for determining the transformation type of the current decoding block. For example, if the value of the first flag is 0, it is determined that the transformation type of the current decoding block is 0, thereby improving the speed of determining the transformation type. , improve decoding efficiency.

Figure 7 is a schematic flowchart of an image decoding method provided by an embodiment of the present application. Figure 7 can be understood as a specific implementation of the decoding method shown in Figure 6.

As shown in Figure 7, the decoding method in this embodiment of the present application includes:

S701. Decode the code stream and obtain the first flag.

In the embodiment of this application, the image area is the current image frame as an example.

In addition, the embodiment of this application takes MTS technology as an example.

At this time, the above-mentioned first flag is used to indicate whether the current image frame is allowed to perform an MTS operation.

In the embodiment of the present application, the encoding end writes the first flag into the code stream, so that after the decoding end obtains the code stream, it decodes the code stream to obtain the first flag.

S702. Decode the code stream to obtain the transform coefficient of the current decoded block.

For example, if the encoding end is encoding, the transform coefficients are not quantized, but the transform coefficients are directly encoded to obtain a code stream. In this way, the decoding end decodes the code stream and can directly obtain the transform coefficient of the current decoding block from the code stream.

For example, when encoding, the encoding end quantizes the transform coefficients to obtain quantized coefficients, and then encodes the quantized coefficients to obtain a code stream. In this way, the decoder decodes the code stream to obtain the quantized coefficients of the current decoded block, and performs inverse quantization on the quantized coefficients to obtain the transform coefficients of the current decoded block.

S703. Determine the transformation type of the current decoding block according to the first flag.

The implementation process of the above S703 is consistent with the above S603. Refer to the description of the above S603 and will not be repeated here.

S704. Determine the transformation matrix of the current decoding block according to the transformation type of the current decoding block and the size of the current decoding block.

S705. Use the transformation matrix to perform an inverse transformation operation on the transformation coefficient to obtain the residual value of the current decoding block.

S706. Determine the prediction value of the current decoding block.

S707: Obtain the reconstruction value of the current decoding block based on the prediction value and residual value of the current decoding block.

The decoding method in the embodiment of the present application obtains the first identifier of the current image frame by decoding the code stream. According to the first identifier, the transformation type of the current decoding block is determined; according to the transformation type of the current decoding block, the transformation of the current decoding block is The coefficients are inversely transformed to obtain the residual value of the current decoded block. That is, the embodiment of the present application adds a condition for determining the transformation type of the current decoding block. For example, if the value of the first flag is 0, it is determined that the transformation type of the current decoding block is 0, thereby improving the speed of determining the transformation type. , improve decoding efficiency.

It should be understood that Figures 4 to 7 are only examples of the present application and should not be understood as limitations of the present application.

The preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings. However, the present application is not limited to the specific details of the above-mentioned embodiments. Within the scope of the technical concept of the present application, various simple modifications can be made to the technical solutions of the present application. These simple modifications all belong to the protection scope of this application. For example, each specific technical feature described in the above-mentioned specific embodiments can be combined in any suitable way without conflict. In order to avoid unnecessary repetition, this application will no longer describe various possible combinations. Specify otherwise. For another example, any combination of various embodiments of the present application can be carried out. As long as they do not violate the idea of the present application, they should also be regarded as the contents disclosed in the present application.

It should also be understood that in the various method embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its functions and internal logic, and should not be used in this application. The implementation of the examples does not constitute any limitations. In addition, in the embodiment of this application, the term "and/or" is only an association relationship describing associated objects, indicating that three relationships can exist. Specifically, A and/or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

The method embodiment of the present application is described in detail above with reference to FIGS. 4 to 7 , and the device embodiment of the present application is described in detail below with reference to FIGS. 8 to 11 .

Figure 8 is a schematic block diagram of an image encoding device provided by an embodiment of the present application.

As shown in Figure 8, the image encoding device 10 includes:

Parameter determination unit 11, used to determine the continuity parameter of the image area where the current coding block is located, the continuity parameter being used to indicate the continuity of changes in the pixel points in the image area;

Mode determination unit 12, configured to determine the transformation mode of the current coding block according to the continuity parameter of the image area;

The transformation unit 13 is configured to transform the current coding block according to the transformation method to obtain the transformation coefficient of the current coding block.

In some embodiments, the parameter determination unit 11 is specifically configured to determine the gradient map of the image area and the gradient map of the gradient map; determine the continuity parameters of the image area based on the gradient map of the gradient map. .

In some embodiments, the parameter determination unit 11 is specifically configured to determine the average value of the gradient values in the gradient map of the gradient map as the continuity parameter of the image area.

In some embodiments, the parameter determination unit 11 is specifically configured to determine the residual value of the image area; determine the continuity parameter of the image area according to the residual value of the image area, wherein the residual value Negatively related to the continuity parameter.

In some embodiments, the mode determination unit 12 is specifically configured to determine the transformation mode of the current coding block to be the first transformation mode if the continuity parameter of the image area is less than a preset threshold.

In some embodiments, the mode determination unit 12 determines the transformation mode of the current coding block from M candidate transformation modes if the continuity parameter of the image area is greater than or equal to the preset threshold, and the M is greater than or equal to A positive integer of 1.

In some embodiments, the mode determination unit 12 determines a first flag according to the continuity parameter of the image area, and the first flag is used to indicate whether the image area is allowed to determine a transformation mode from M candidate transformation modes. , the M is a positive integer greater than 1; according to the first flag, the transformation mode of the current coding block is determined.

In some embodiments, the mode determination unit 12 determines that the value of the first flag is a first value if the continuity parameter of the image area is less than a preset threshold, and the first value is used to indicate that it is not allowed. The image area determines a transformation mode from the M candidate transformation modes.

In some embodiments, the mode determining unit 12 determines that the transformation mode of the current coding block is the first transformation mode if the value of the first flag is the first numerical value.

In some embodiments, the mode determination unit 12 determines that the value of the first flag is a second value if the continuity parameter of the image area is greater than or equal to the preset threshold, and the second value is The instruction allows the image region to determine a transformation mode from the M candidate transformation modes.

In some embodiments, the mode determining unit 12 determines the transformation mode of the current coding block from M candidate transformation modes if the value of the first flag is the second value.

In some embodiments, the mode determination unit 12 determines the rate distortion cost when using the M candidate transformation modes to transform the current coding block; according to the rate distortion costs corresponding to the M candidate transformation modes, from the Select one candidate transformation mode from the M candidate transformation modes as the transformation mode of the current coding block.

In some embodiments, the mode determination unit 12 determines the candidate transformation mode with the smallest rate distortion cost among the M candidate transformation modes as the transformation mode of the current coding block.

In some embodiments, the transformation unit 13 is used to write the first flag into the code stream.

In some embodiments, the image area is the current image frame, or the current image sequence, or the current image slice, or the current coding block.

In some embodiments, if the image area is any one of the current image frame, the current image slice, and the current coding block, the code stream further includes a second flag, and the third The second flag is used to indicate whether the current picture sequence allows the transformation mode to be determined from the M candidate transformation modes.

Optionally, if the image area is the current image sequence, the first mark multiplexes the second mark.

Optionally, the first conversion mode is DTC-II mode.

Optionally, the M candidate transformation modes include a transformation mode composed of a combination of DTC-II mode, DST-VII mode and DCT-VIII mode.

Optionally, the M candidate transformation modes include multiple transformation modes corresponding to the multi-transform core selection MTS.

It should be understood that the device embodiments and the method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device 10 shown in Figure 8 can execute the encoding method of the embodiment of the present application, and perform the aforementioned and other operations and/or functions of each unit in the device 10 in order to implement the corresponding processes in each method such as the above encoding method. For the sake of brevity, no further details will be given here.

Figure 9 is a schematic block diagram of an image decoding device provided by an embodiment of the present application.

As shown in Figure 9, the image decoding device 20 includes:

Decoding unit 21, used to decode the code stream and determine the transform coefficient of the current decoding block;

The flag determination unit 22 is used to determine the first flag of the image area where the current encoding block is located. The first flag is determined based on the continuity parameter of the image area. The first flag is used to indicate whether the The image area determines the transformation mode from M candidate transformation modes, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;

Type determination unit 23, configured to determine the transformation type of the current decoding block according to the first flag;

The inverse transform unit 24 is configured to inversely transform the transform coefficient of the current decoded block according to the transform type of the current decoded block to obtain the residual value of the current decoded block.

In some embodiments, the flag determining unit 22 is specifically configured to determine the continuity parameter of the image area; determine the first flag according to the continuity parameter of the image area.

In some embodiments, the flag determining unit 22 is specifically configured to determine the adjacent decoded area of the image area; determine the gradient map of the adjacent decoded area, and the gradient map of the gradient map; according to the A gradient map of a gradient map that determines the continuity parameters of the image region.

In some embodiments, the flag determining unit 22 is specifically configured to determine the average value of the gradient values in the gradient map of the gradient map as the continuity parameter of the image area.

In some embodiments, the flag determining unit 22 is specifically configured to determine adjacent decoded areas of the image area; determine the residual value of the adjacent decoded area; and determine the residual value of the adjacent decoded area according to the residual value of the adjacent decoded area. value to determine the continuity parameter of the image region, wherein the residual value is negatively correlated with the continuity parameter.

In some embodiments, the flag determination unit 22 is specifically configured to determine that the value of the first flag is a first value if the continuity parameter of the image area is less than a preset threshold, and the first value is used for Indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes.

In some embodiments, the flag determination unit 22 is specifically configured to determine that the value of the first flag is a second value if the continuity parameter of the image area is greater than or equal to a preset threshold, and the second value Used to indicate that the image area is allowed to determine the transformation mode from the M candidate transformation modes.

In some embodiments, the flag determining unit 22 is specifically configured to decode the code stream to obtain the first flag.

In some embodiments, the type determination unit 23 is configured to determine that the index of the transform type of the current decoding block is 0 if the value of the first flag is a first value, and the first value is used for Indicates that the image area is not allowed to determine the transformation mode from the M candidate transformation modes.

In some embodiments, the type determination unit 23 is configured to determine the transformation type of the current decoding block according to the second flag if the value of the first flag is a second value, and the second The flag is obtained by decoding the code stream. The second value is used to indicate that the image area is allowed to determine the transformation mode from the M candidate transformation modes. The second flag is used to indicate whether the current picture sequence is allowed to be transformed from all the candidate transformation modes. The transformation method is determined among the M candidate transformation methods.

Optionally, the transformation type includes a horizontal transformation type and a vertical transformation type.

Optionally, the image area is the current image frame, or the current image sequence, or the current image slice, or the current coding block.

Optionally, if the image area is the current image sequence, the second mark and the first mark are the same mark.

In some embodiments, the inverse transformation unit 24 is configured to determine a transformation matrix according to the transformation type of the current decoding block and the size of the current decoding block; use the transformation matrix to perform an inverse transformation operation on the transformation coefficient, Obtain the residual value of the current decoding block.

It should be understood that the device embodiments and the method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device 20 shown in FIG. 9 may correspond to the corresponding subject in performing the decoding method of the embodiment of the present application, and the foregoing and other operations and/or functions of each unit in the device 20 are respectively to implement the decoding method and other methods. The corresponding process in , for the sake of brevity, will not be repeated here.

The device and system of the embodiments of the present application are described above from the perspective of functional units in conjunction with the accompanying drawings. It should be understood that this functional unit can be implemented in the form of hardware, can also be implemented in the form of instructions in the software, or can also be implemented in a combination of hardware and software units. Specifically, each step of the method embodiments in the embodiments of the present application can be completed by integrated logic circuits of hardware in the processor and/or instructions in the form of software. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly embodied in hardware. The execution of the decoding processor is completed, or the execution is completed using a combination of hardware and software units in the decoding processor. Optionally, the software unit may be located in a mature storage medium in this field such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in combination with its hardware.

Figure 10 is a schematic block diagram of an electronic device provided by an embodiment of the present application.

As shown in Figure 32, the electronic device 30 may be the video encoder or video decoder described in the embodiment of the present application. The electronic device 30 may include:

Memory 33 and processor 32, the memory 33 is used to store the computer program 34 and transmit the program code 34 to the processor 32. In other words, the processor 32 can call and run the computer program 34 from the memory 33 to implement the method in the embodiment of the present application.

For example, the processor 32 may be configured to perform the steps in the above method 200 according to instructions in the computer program 34 .

In some embodiments of the present application, the processor 32 may include but is not limited to:

General processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components, etc.

In some embodiments of the present application, the memory 33 includes but is not limited to:

Volatile memory and/or non-volatile memory. Among them, non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory may be Random Access Memory (RAM), which is used as an external cache. By way of illustration, but not limitation, many forms of RAM are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

In some embodiments of the present application, the computer program 34 can be divided into one or more units, and the one or more units are stored in the memory 33 and executed by the processor 32 to complete the tasks provided by this application. Methods. The one or more units may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the execution process of the computer program 34 in the electronic device 30 .

As shown in Figure 10, the electronic device 30 may also include:

Transceiver 33 , the transceiver 33 can be connected to the processor 32 or the memory 33 .

The processor 32 can control the transceiver 33 to communicate with other devices. Specifically, it can send information or data to other devices, or receive information or data sent by other devices. Transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include an antenna, and the number of antennas may be one or more.

It should be understood that various components in the electronic device 30 are connected through a bus system, where in addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus.

As shown in Figure 11, the video encoding and decoding system 40 may include: a video encoder 41 and a video decoder 42, where the video encoder 41 is used to perform the video encoding method involved in the embodiment of the present application, and the video decoder 42 is used to perform The embodiment of the present application relates to a video decoding method.

This application also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, embodiments of the present application also provide a computer program product containing instructions, which when executed by a computer causes the computer to perform the method of the above method embodiments.

This application also provides a code stream, which is generated by the above encoding method. Optionally, the code stream includes a first flag.

When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted over a wired connection from a website, computer, server, or data center (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The available media may be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as digital video discs (DVD)), or semiconductor media (such as solid state disks (SSD)), etc.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separate. A component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, each functional unit in various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

The above contents are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, and should are covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image coding method, characterized by including:

Determine the continuity parameter of the image area where the current coding block is located, the continuity parameter being used to indicate the continuity of changes in pixels in the image area;

Determine the transformation mode of the current coding block according to the continuity parameter of the image area;

According to the transformation method, the current coding block is transformed to obtain the transformation coefficient of the current coding block.
The method according to claim 1, characterized in that determining the continuity parameter of the image area where the current coding block is located includes:

Determine the gradient map of the image area, and the gradient map of the gradient map;

A continuity parameter of the image region is determined based on the gradient map of the gradient map.
The method of claim 2, wherein determining the continuity parameter of the image area according to the gradient map of the gradient map includes:

The average value of the gradient values in the gradient map of the gradient map is determined as the continuity parameter of the image area.
The method according to claim 1, characterized in that determining the continuity parameter of the image area where the current coding block is located includes:

Determine the residual value of the image area;

A continuity parameter of the image region is determined based on a residual value of the image region, wherein the residual value is negatively correlated with the continuity parameter.
The method according to any one of claims 1 to 4, characterized in that determining the transformation mode of the current coding block according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is less than the preset threshold, it is determined that the transformation mode of the current coding block is the first transformation mode.
The method according to any one of claims 1 to 4, characterized in that determining the transformation mode of the current coding block according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is greater than or equal to the preset threshold, then the transformation mode of the current coding block is determined from M candidate transformation modes, where M is a positive integer greater than 1.
The method according to any one of claims 1 to 4, characterized in that determining the transformation mode of the current coding block according to the continuity parameter of the image area includes:

Determine a first flag according to the continuity parameter of the image area. The first flag is used to indicate whether the image area is allowed to determine a transformation mode from M candidate transformation modes, where M is a positive integer greater than 1;

According to the first flag, the transformation mode of the current coding block is determined.
The method of claim 7, wherein determining the first mark according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is less than the preset threshold, it is determined that the value of the first flag is a first value, and the first value is used to indicate that the image area is not allowed to transform from the M candidates. Determine the transformation method in the method.
The method of claim 8, wherein determining the transformation mode of the current coding block according to the first flag includes:

If the value of the first flag is the first numerical value, it is determined that the transformation mode of the current coding block is the first transformation mode.
The method of claim 7, wherein determining the first mark according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is greater than or equal to the preset threshold, it is determined that the value of the first flag is a second value, and the second value is used to indicate that the image area is allowed to move from the M Determine the transformation method among candidate transformation methods.
The method of claim 10, wherein determining the transformation mode of the current coding block according to the first flag includes:

If the value of the first flag is the second value, the transformation mode of the current coding block is determined from M candidate transformation modes.
The method according to claim 6 or 11, characterized in that determining the transformation mode of the current coding block from M candidate transformation modes includes:

Determine the rate distortion cost when using the M candidate transformation methods to transform the current coding block;

According to the rate distortion cost corresponding to the M candidate transformation modes, a candidate transformation mode is selected from the M candidate transformation modes as the transformation mode of the current coding block.
The method according to claim 12, characterized in that, according to the rate distortion cost corresponding to the M candidate transformation modes, a candidate transformation mode is selected from the M candidate transformation modes as the current coding block. Transformation methods include:

Among the M candidate transformation modes, the candidate transformation mode with the smallest rate distortion cost is determined as the transformation mode of the current coding block.
The method of claim 7, further comprising:

Write the first flag into the code stream.
The method of claim 14, wherein the image area is a current image frame, a current image sequence, a current image slice, or the current coding block.
The method according to claim 15, characterized in that if the image area is any one of the current image frame, the current image slice and the current coding block, the code stream also includes The second flag is used to indicate whether the current image sequence allows the transformation mode to be determined from the M candidate transformation modes.
The method according to claim 16, characterized in that if the image area is the current image sequence, the first mark multiplexes the second mark.
The method according to claim 5, characterized in that the first transformation method is a discrete cosine transform DTC-II method.
The method according to claim 6, wherein the M candidate transformation methods include a combination of discrete cosine transform DTC-II, discrete sine transform DST-VII, and discrete cosine transform DCT-VIII.
The method according to claim 19, characterized in that the M candidate transformation modes include multiple transformation modes corresponding to multi-transform core selection MTS.
An image decoding method, characterized by including:

Decode the code stream and determine the transform coefficient of the current decoding block;

Determine the first flag of the image area where the current coding block is located, the first flag is determined based on the continuity parameter of the image area, and the first flag is used to indicate whether the image area is allowed to transform from M candidates The transformation mode is determined in the mode, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;

Determine the transform type of the current decoding block according to the first flag;

According to the transformation type of the current decoding block, the transformation coefficient of the current decoding block is inversely transformed to obtain the residual value of the current decoding block.
The method according to claim 21, characterized in that determining the first flag of the image area where the current coding block is located includes:

Determine continuity parameters of the image area;

The first flag is determined based on the continuity parameter of the image area.
The method of claim 22, wherein determining the continuity parameter of the image area includes:

Determine adjacent decoded areas of the image area;

Determine the gradient map of the adjacent decoded region, and the gradient map of the gradient map;

A continuity parameter of the image region is determined based on the gradient map of the gradient map.
The method of claim 23, wherein determining the continuity parameter of the image area according to the gradient map of the gradient map includes:

The average value of the gradient values in the gradient map of the gradient map is determined as the continuity parameter of the image area.
The method of claim 22, wherein determining the continuity parameter of the image area includes:

Determine adjacent decoded areas of the image area;

Determine the residual value of the adjacent decoded region;

The continuity parameter of the image region is determined according to the residual value of the adjacent decoded region, wherein the residual value is negatively correlated with the continuity parameter.
The method according to any one of claims 22 to 25, characterized in that determining the first mark according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is less than the preset threshold, it is determined that the value of the first flag is a first value, and the first value is used to indicate that the image area is not allowed to transform from the M candidates. Determine the transformation method in the method.
The method according to any one of claims 22 to 25, characterized in that determining the first mark according to the continuity parameter of the image area includes:

If the continuity parameter of the image area is greater than or equal to the preset threshold, the value of the first flag is determined to be a second value, and the second value is used to indicate that the image area is allowed to be selected from the M candidates. Determine the transformation method in the transformation method.
The method according to claim 21, characterized in that determining the first flag of the image area where the current coding block is located includes:

Decode the code stream to obtain the first flag.
The method of claim 21, wherein determining the transformation type of the current decoding block according to the first flag includes:

If the value of the first flag is a first value, it is determined that the transformation type index of the current decoding block is 0, and the first value is used to indicate that the image area is not allowed to transform from the M candidates. Determine the transformation method in the method.
The method of claim 21, wherein determining the transformation type of the current decoding block according to the first flag includes:

If the value of the first flag is a second value, the transformation type of the current decoding block is determined according to the second flag. The second flag is obtained by parsing the code stream. The second value is In order to indicate that the image region is allowed to determine a transformation mode from the M candidate transformation modes, the second flag is used to indicate whether the current image sequence is allowed to determine a transformation mode from the M candidate transformation modes.
The method of claim 30, wherein the transformation type includes a horizontal transformation type and a vertical transformation type.
The method according to claim 31, characterized in that the image area is a current image frame, a current image sequence, a current image slice, or the current coding block.
The method according to claim 32, characterized in that if the image area is the current image sequence, the second mark and the first mark are the same mark.
The method according to claim 21, characterized in that, according to the transformation type of the current decoding block, the transformation coefficient of the current decoding block is inversely transformed to obtain the residual value of the current decoding block, including :

Determine a transformation matrix according to the transformation type of the current decoding block and the size of the current decoding block;

Use the transformation matrix to perform an inverse transformation operation on the transformation coefficient to obtain the residual value of the current decoding block.
The method according to claim 21, characterized in that the M candidate transformation modes include a transformation mode composed of a combination of discrete cosine transform DTC-II mode, discrete sine transform DST-VII mode and DCT-VIII mode.
The method according to claim 35, characterized in that the M candidate transformation modes include multiple transformation modes corresponding to multi-transform core selection MTS.
An image coding device, characterized in that it includes:

A parameter determination unit, used to determine the continuity parameter of the image area where the current coding block is located, the continuity parameter being used to indicate the continuity of changes in the pixel points in the image area;

A mode determination unit, configured to determine the transformation mode of the current coding block according to the continuity parameter of the image area;

A transformation unit, configured to transform the current coding block according to the transformation method to obtain the transformation coefficient of the current coding block.
An image decoding device, characterized by including:

The decoding unit is used to decode the code stream and determine the transform coefficient of the current decoding block;

A flag determination unit, used to determine a first flag of the image area where the current encoding block is located, the first flag is determined based on the continuity parameter of the image area, and the first flag is used to indicate whether the image is allowed The region determines the transformation method from M candidate transformation methods, the continuity parameter is used to indicate the continuity of changes in the pixel points in the image area, and the M is a positive integer greater than 1;

A type determination unit, configured to determine the transformation type of the current decoding block according to the first flag;

An inverse transform unit, configured to perform inverse transform on the transform coefficient of the current decoded block according to the transform type of the current decoded block, to obtain the residual value of the current decoded block.
A video encoder, characterized by including a processor and a memory;

The memory shown is used to store computer programs;

The processor is configured to call and run the computer program stored in the memory to implement the method as described in any one of claims 1 to 20.
A video decoder, characterized by including a processor and a memory;

The memory shown is used to store computer programs;

The processor is configured to call and run the computer program stored in the memory to implement the method as described in any one of claims 21 to 36.
A coding and decoding system, characterized by including:

A video encoder according to claim 39;

And a video decoder according to claim 40.
A computer-readable storage medium, characterized in that it is used to store a computer program;

The computer program causes the computer to perform the method according to any one of claims 1 to 20 or 21 to 36 above.
A code stream, characterized in that the code stream is generated by the method described in any one of claims 1 to 20 or 21 to 36.