WO2023083263A1

WO2023083263A1 - Video encoding and decoding method and apparatus

Info

Publication number: WO2023083263A1
Application number: PCT/CN2022/131167
Authority: WO
Inventors: 曹小强; 孙煜程; 陈方栋; 王莉
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2021-11-11
Filing date: 2022-11-10
Publication date: 2023-05-19
Also published as: TWI821013B; TW202327364A; CN116112689A

Abstract

The present application relates to the technical field of video encoding and decoding, and discloses a video encoding and decoding method and apparatus, which are used for providing a new transformation method for current encoding and decoding technologies. The transformation method can improve the transformation flexibility. The use of the transformation method based on different dimensions can reduce the calculation complexity, reduce the calculation burden of a hardware device, reduce the hardware implementation complexity, and improve the compression performance. The decoding method comprises: performing entropy decoding and inverse quantization on a code stream to obtain an inverse transform unit; and performing discrete wavelet inverse transform on the inverse transform unit to obtain an inverse transform coefficient.

Description

Video encoding and decoding method and device

Related Application Cross Reference

This patent application claims priority to Chinese patent application No. 202111335612.2 filed on November 11, 2021, which is incorporated herein by reference in its entirety.

technical field

The present application relates to the technical field of video coding and decoding, and in particular to a video coding and decoding method and device.

Background technique

Video is an important way for users to obtain information. In the process of video storage and transmission, considering the quality of information and the occupied space, it is necessary to compress the video to balance the relationship between information integrity and occupied space. Video coding (Video Coding) technology is a way to achieve compression. Video coding mainly obtains code streams through the steps of prediction, transformation, quantization and entropy coding. The current transformation method mainly adopts a preset transformation method for the transformation unit in the horizontal direction and vertical direction, wherein the preset transformation method is discrete cosine transform (Discrete Cosine Transform, DCT) or a conceptually similar transformation (for example, Discrete Sine Transform (Discrete Sine Transform, DST)), and the flexibility of the above-mentioned method is low, and the calculation complexity of the above two transformation methods is high, and it is not convenient to realize video encoding and decoding.

Contents of the invention

Embodiments of the present application provide a video encoding and decoding method and device, which improve the flexibility of a transformation method, reduce computational complexity, and improve compression performance.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions: In the first aspect, the embodiment of the present application provides a video decoding method, which is applied to a video decoding device or a video decoding device, and the method includes: performing Entropy decoding and inverse quantization are used to obtain an inverse transform unit; discrete wavelet inverse transform is performed on the inverse transform unit to obtain inversely transformed coefficients.

Due to the high computational complexity of the current transformation method, the inverse transformation unit based on the current large size needs more resources in the calculation process, which increases the complexity of hardware implementation. This application proposes to use discrete wavelet inverse transform method for video decoding. The discrete wavelet inverse transform can remove the correlation between the internal pixels of each sub-image, and concentrate the information on as few inverse transform coefficients as possible, so that the next quantization step may quantize the coefficients carrying less information to 0, Reduce the impact on the reconstructed image quality. The discrete wavelet inverse transform is used to greatly reduce the computational complexity, correspondingly, the burden of computational storage is reduced, and the complexity of hardware implementation is reduced.

In a possible implementation manner, performing discrete wavelet inverse transform on the inverse transform unit to obtain the coefficients after inverse transform includes: performing discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and performing discrete wavelet inverse transform in the vertical direction transformation, DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation; or, perform any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation on the inverse transformation unit in the horizontal direction, Inverse discrete wavelet transform is performed in the vertical direction.

This possible implementation provides three ways of performing inverse discrete wavelet transform in the horizontal direction and/or vertical direction, and performing inverse discrete wavelet transform in only one direction helps to reduce computational complexity. The same or different inverse transformation methods can be used for the two directions, which improves the flexibility of the inverse transformation process and is compatible with existing algorithms.

In a possible implementation, the discrete wavelet inverse transform includes Haar wavelet inverse transform and 5/3 wavelet inverse transform, and the inverse discrete wavelet transform is performed on the inverse transform unit to obtain the coefficients after the inverse transform, including: The transformation unit performs inverse Haar wavelet transform in the horizontal direction, and performs any one of inverse Haar wavelet transform, 5/3 wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction; or, for The inverse transformation unit performs 5/3 wavelet inverse transformation in the horizontal direction, and performs any one of Haar wavelet inverse transformation, 5/3 wavelet inverse transformation, DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation in the vertical direction Or, carry out any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation to the described inverse transformation unit in the horizontal direction, and carry out Haar wavelet inverse transformation in the vertical direction; Or, to the described inverse transformation unit in Perform any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation in the horizontal direction, and perform 5/3 wavelet inverse transformation in the vertical direction.

This possible implementation method provides two inverse transform methods including the discrete wavelet inverse transform, combining the horizontal direction and the vertical direction to provide multiple combination methods. According to the translation and stretching of the Haar wavelet inverse transform and the 5/3 wavelet inverse transform, the calculation of multiplication is avoided, which helps to reduce the computational burden when processing large-scale data, and at the same time reduce the computational resources occupied during the calculation process .

In a possible implementation manner, the size of the inverse transformation unit is the same as the size of the coding unit; or, the width of the inverse transformation unit is half of the width of the coding unit, and the height of the inverse transformation unit is the same as the height of the coding unit ; or, the height of the inverse transformation unit is half of the height of the coding unit, and the width of the inverse transformation unit is the same as the width of the coding unit.

This possible implementation method provides a way to determine the size of the inverse transformation unit, and gives the relationship between the size of the inverse transformation unit and the size of the coding unit, and further dividing the coding unit during the transformation helps to reduce the computational complexity. .

A possible implementation manner is to perform discrete wavelet inverse transform on the inverse transform unit to obtain the coefficients after the inverse transform, including: based on the size of the inverse transform unit, determine to perform the first inverse transform on the inverse transform unit in the horizontal direction, and then The second inverse transformation is performed in the vertical direction; wherein, at least one of the first inverse transformation and the second inverse transformation is discrete wavelet inverse transformation.

This possible implementation method provides inverse discrete wavelet transform based on the horizontal direction and/or vertical direction. When the inverse transformation is performed based on one dimension, it helps to greatly reduce the computational burden and improve the compression performance.

In a possible implementation, the size of the inverse transformation unit includes a first size and a second size, and based on the size of the inverse transformation unit, it is determined to perform the first inverse transformation on the inverse transformation unit in the horizontal direction, and perform the second inverse transformation in the vertical direction. The inverse transform includes: based on the first size, determining to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and performing discrete wavelet inverse transform on the vertical direction; based on the second size, determining to perform the inverse transform unit on the horizontal direction Inverse discrete wavelet transform is performed, and no inverse transform is performed in the vertical direction.

This possible implementation provides a specific implementation of the corresponding transformation method based on the size of the inverse transformation unit, wherein the discrete wavelet transformation can be selected not to be performed in the vertical direction, thereby reducing the calculation burden.

In a possible implementation, the inverse discrete wavelet transform is performed on the inverse transform unit, and before the inversely transformed coefficients are obtained, the method further includes: acquiring a transformation method identification bit in the code stream, and the transformation method identification bit is used for Indicates the inverse transform method of the inverse transform unit, the inverse transform method includes performing discrete wavelet inverse transform in the horizontal direction and/or vertical direction.

This possible implementation provides a specific implementation of determining the inverse transformation method based on the transformation method identification bit in the code stream. The way of direct indication helps to improve the efficiency of decoding, and at the same time establishes the corresponding relationship between the encoding side and the decoding side.

In a possible implementation manner, before performing discrete wavelet inverse transform on the inverse transform unit to obtain the inversely transformed coefficients, the method further includes: acquiring a prediction mode identification bit in the code stream, where the prediction mode identification bit is used to indicate the Before the inverse transformation unit performs inverse transformation, the prediction mode for the prediction unit to perform prediction, the prediction mode includes a first prediction mode and a second prediction mode; based on the first prediction mode, it is determined that the inverse transformation unit is discretely performed in the horizontal direction Inverse wavelet transform, performing any one of discrete wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction; based on the second prediction mode, it is determined to perform DCT2 inverse transform on the inverse transform unit in the horizontal direction Transform, DST7 inverse transform and no inverse transform, perform discrete wavelet inverse transform in the vertical direction.

This possible implementation provides a specific implementation of determining the inverse transformation method based on the prediction mode identification bit in the code stream. By establishing the corresponding relationship between the prediction mode and the inverse transformation method, the inverse transformation method is determined according to the effect of different prediction modes, the flexibility of the inverse transformation is enhanced, and the compression effect is improved. According to the predefined relationship, it helps to improve the efficiency of decoding.

A possible implementation, the prediction mode includes any one of DC prediction mode, angle prediction mode, Planar prediction mode and block copy mode, and the angle prediction mode includes any of horizontal prediction mode, vertical prediction mode and diagonal prediction mode A sort of.

This possible implementation method provides a type of prediction mode, which is used to establish a corresponding relationship with a transformation method and improve decoding efficiency.

A possible implementation method is to perform discrete wavelet inverse transform on the inverse transform unit to obtain inversely transformed coefficients, including: performing discrete wavelet inverse transform on the brightness of the inverse transform unit, and performing discrete wavelet inverse transform on the chroma of the inverse transform unit , any one of DCT2 inverse transform and DST7 inverse transform; or, perform any one of DCT2 inverse transform and DST7 inverse transform on the brightness of the inverse transform unit, and perform discrete wavelet inverse transform on the chrominance of the inverse transform unit.

This possible implementation method provides inverse discrete wavelet transform based on brightness and chroma, where the inverse transform based on brightness and chroma can be performed in combination with the size of brightness and chroma, which helps to reduce computational complexity and improve compression performance.

In the second aspect, the embodiment of the present application provides a video decoding method, which is applied to a video decoding device or a video decoding device, and the method includes: performing entropy decoding and inverse quantization on a code stream to obtain an inverse transformation unit; Perform the third inverse transformation to obtain the coefficients after the inverse transformation; wherein, the third inverse transformation includes inverse transformation in the horizontal direction and/or inverse transformation in the vertical direction, and the inverse transformation methods in the horizontal direction and the vertical direction include DCT2 inverse transformation or DST7 inverse transform.

Due to the high computational complexity of the current inverse transformation method, using different inverse transformation methods based on different dimensions can help reduce computational complexity and improve compression performance.

In a possible implementation manner, the size of the inverse transformation unit is the same as the size of the coding unit; or, the width of the inverse transformation unit is half of the width of the coding unit, and the height of the inverse transformation unit is the same as that of the coding unit or, the height of the inverse transformation unit is half of the height of the coding unit, and the width of the inverse transformation unit is the same as the width of the coding unit.

This possible implementation method provides a way to determine the size of the inverse transformation unit, and gives the relationship between the size of the inverse transformation unit and the size of the coding unit, and further dividing the coding unit during inverse transformation helps to reduce computational complexity Spend.

In a possible implementation, the size of the inverse transformation unit includes a third size, a fourth size, a fifth size, and a sixth size, and the third inverse transformation is performed on the inverse transformation unit to obtain the coefficient after the inverse transformation, including: based on the inverse For the third size of the transformation unit, DCT2 inverse transformation or DST7 inverse transformation is performed on the inverse transformation unit in the horizontal direction, and inverse transformation is not performed in the vertical direction; based on the fourth size of the inverse transformation unit, inverse transformation is not performed on the inverse transformation unit in the horizontal direction Transformation, perform DCT2 inverse transformation or DST7 inverse transformation in the vertical direction; based on the fifth size of the inverse transformation unit, perform DCT2 inverse transformation in the horizontal direction on the inverse transformation unit, and perform DST7 inverse transformation in the vertical direction; based on the sixth size of the inverse transformation unit Size, perform DST7 inverse transformation on the inverse transformation unit in the horizontal direction, and perform DCT2 inverse transformation in the vertical direction.

This possible implementation provides a correspondence between the size of the inverse transformation unit and the inverse transformation method. When inverse transformation is performed based on one dimension, it helps to greatly reduce the computational burden and improve compression performance.

In a possible implementation, the third inverse transform is performed on the inverse transform unit, and before the inversely transformed coefficients are obtained, the method further includes: acquiring a transform method identification bit in the code stream, where the transform method identification bit is used to indicate the inverse transform An inverse transform method of the transform unit; determining a third inverse transform according to the inverse transform method of the inverse transform unit.

In a possible implementation, the inverse discrete wavelet transform is performed on the inverse transform unit to obtain the inversely transformed coefficients. Before the transformation unit performs inverse transformation, the prediction mode for the prediction unit to perform prediction, the prediction mode includes the third prediction mode, the fourth prediction mode, the fifth prediction mode, and the sixth prediction mode; based on the third prediction mode, determine the inverse transformation The unit performs DCT2 inverse transformation or DST7 inverse transformation in the horizontal direction, and does not perform transformation in the vertical direction; based on the fourth prediction mode, it is determined that the inverse transformation unit does not perform inverse transformation in the horizontal direction, and performs DCT2 inverse transformation or DST7 in the vertical direction Inverse transformation; based on the fifth prediction mode, perform DCT2 inverse transformation on the inverse transformation unit in the horizontal direction, and perform DST7 inverse transformation on the vertical direction; based on the sixth prediction mode, perform DST7 inverse transformation on the inverse transformation unit in the horizontal direction , perform DCT2 inverse transformation in the vertical direction.

In a possible implementation manner, the prediction mode includes any one of DC mode, angle mode, Planar mode, and block copy mode, and the angle mode is any one of horizontal mode, vertical mode, and diagonal mode.

This possible implementation provides a type of prediction mode, which is used to establish a corresponding relationship with the inverse transform method and improve decoding efficiency.

A possible implementation method is to perform a third inverse transformation on the inverse transformation unit to obtain the coefficients after the inverse transformation, including: performing DCT2 inverse transformation or DST7 inverse transformation on the brightness of the inverse transformation unit; performing DCT2 on the chroma of the inverse transformation unit Inverse transformation or DST7 inverse transformation to obtain the coefficients after inverse transformation.

This possible implementation method provides inverse transformation based on luminance and chrominance respectively. The inverse transformation based on luminance and chrominance can be performed in combination with the size of luminance and chrominance, which helps to reduce computational complexity and improve compression performance. .

In a third aspect, the embodiment of the present application provides a video coding method, which is applied to a video coding device or a video coding device, and the method includes: performing discrete wavelet transform on a transform unit to obtain a transform coefficient; performing quantization on the transform coefficient and Entropy coding to obtain code stream.

In a fourth aspect, an embodiment of the present application provides a video coding method, which is applied to a video coding device or a video coding device, and the method includes: performing a third transformation on a transformation unit to obtain a transformation coefficient; wherein, the third transformation includes Performing transformation in the horizontal direction and/or performing transformation in the vertical direction, the transformation methods in the horizontal direction and the vertical direction include DCT2 transformation or DST7 transformation; performing quantization and entropy coding on the transformation coefficients to obtain a code stream.

In a fifth aspect, an embodiment of the present application provides a video decoding device, which has a function of implementing the video decoding method in any one of the foregoing first aspects. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In a sixth aspect, an embodiment of the present application provides a video decoding device, which has a function of implementing the video decoding method in any one of the above-mentioned second aspects. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In a seventh aspect, an embodiment of the present application provides a video encoding device, which has a function of implementing the video encoding method in any one of the above third aspects. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In an eighth aspect, an embodiment of the present application provides a video encoding device, which has a function of implementing the video encoding method in any one of the foregoing fourth aspects. This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware. The hardware or software includes one or more modules corresponding to the above functions.

In a ninth aspect, there is provided a video decoding device, including: a processor and a memory; the memory is used to store computer-executable instructions, and when the video decoding device is running, the processor executes the computer-executable instructions stored in the memory, so that The video decoding device executes the video decoding method according to any one of the first aspect above.

In a tenth aspect, a video decoding device is provided, including: a processor and a memory; the memory is used to store computer-executable instructions, and when the video decoding device is running, the processor executes the computer-executable instructions stored in the memory, so that The video decoding device executes the video decoding method according to any one of the second aspect above.

In an eleventh aspect, a video encoding device is provided, including: a processor and a memory; the memory is used to store computer-executable instructions, and when the video encoding device is running, the processor executes the computer-executable instructions stored in the memory to Make the video coding device execute the video coding method according to any one of the third aspect above.

In a twelfth aspect, a video encoding device is provided, including: a processor and a memory; the memory is used to store computer-executable instructions, and when the video encoding device is running, the processor executes the computer-executable instructions stored in the memory to Make the video coding device execute the video coding method according to any one of the fourth aspect above.

In a thirteenth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium, and when it is run on a computer, the computer can execute any one of the above-mentioned first to fourth aspects. Video codec method.

In a fourteenth aspect, there is provided a computer program product including instructions, which, when run on a computer, enable the computer to execute the video encoding and decoding method in any one of the first aspect to the fourth aspect.

In a fifteenth aspect, an electronic device is provided, the electronic device includes a video decoding apparatus, and the processing circuit is configured to execute the video decoding method according to any one of the first aspect and the second aspect above.

In a sixteenth aspect, an electronic device is provided, the electronic device includes a video encoding device, and the processing circuit is configured to execute the video encoding method according to any one of the third aspect and the fourth aspect above.

In a seventeenth aspect, an electronic device is provided, the electronic device includes a video decoding device and a video encoding device, and the processing circuit is configured to execute the video decoding method according to any one of the first aspect above and any one of the third aspect Video encoding method.

In an eighteenth aspect, an electronic device is provided, the electronic device includes a video decoding device and a video encoding device, and the processing circuit is configured to execute the video decoding method according to any one of the above-mentioned second aspect and any one of the fourth aspect Video encoding method.

In a nineteenth aspect, a communication system is provided, and the communication system includes the video decoding device in the fifth aspect and the sixth aspect and the video encoding device in the seventh aspect and the eighth aspect in the above aspects.

For the technical effects brought about by any one of the implementation manners from the third aspect to the nineteenth aspect, refer to the technical effects brought about by the corresponding implementation manners in the first aspect and the second aspect, and details are not repeated here.

Description of drawings

Figure 1a is a schematic diagram of a DC prediction mode;

Figure 1b is a schematic diagram of a vertical prediction mode;

Figure 1c is a schematic diagram of the Planar prediction model;

Figure 1d is a schematic diagram of a block copy mode;

FIG. 2 is a system architecture diagram of a codec system provided in an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a video encoder provided in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a video decoder provided in an embodiment of the present application;

FIG. 5 is a schematic flow diagram of a video encoding/decoding provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a video decoder or video encoder provided in an embodiment of the present application;

FIG. 7 is a flowchart of a video decoding method provided in an embodiment of the present application;

Fig. 8 is a schematic diagram of the 5/3 wavelet transform method provided by the embodiment of the present application;

FIG. 9 is a flowchart of a video decoding method provided in an embodiment of the present application;

FIG. 10 is a flowchart of a video encoding method provided in an embodiment of the present application;

FIG. 11 is a flow chart of a video encoding method provided in an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a decoding device provided by an embodiment of the present application;

FIG. 13 is a schematic structural diagram of a decoding device provided by an embodiment of the present application;

FIG. 14 is a schematic structural diagram of an encoding device provided by an embodiment of the present application;

FIG. 15 is a schematic structural diagram of an encoding device provided by an embodiment of the present application.

Detailed ways

In the description of the present application, unless otherwise specified, "/" means "or", for example, A/B may mean A or B. The "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist at the same time, and B exists alone These three situations. In addition, "at least one" means one or more, and "plurality" means two or more. Words such as "first" and "second" do not limit the number and order of execution, and words such as "first" and "second" do not necessarily limit the difference.

It should be noted that, in this application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design described herein as "exemplary" or "for example" is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

First, introduce the technical terms involved in the embodiment of this application:

1. Video coding technology

Video sequences have a series of redundant information such as spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. In order to remove the redundant information in the video sequence as much as possible and reduce the amount of data representing the video, video coding technology is proposed to achieve the effect of reducing storage space and saving transmission bandwidth. Video coding technology is also called video compression technology.

In the international general scope, video compression coding standards, such as: Advanced Video Coding (Advanced Video Coding, AVC), H.263, H.264 and H.265 (also known as High Efficiency Video Coding standard) developed by the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T). , HEVC)).

It should be noted that, in the coding algorithm based on the hybrid coding architecture, the above-mentioned compression coding methods may be used in combination.

The basic processing unit in the process of video compression encoding is an image block, which is obtained by dividing a frame/image at the encoding end. Taking HEVC as an example, HEVC defines Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU) and Transform Unit (TU). Coding tree units, coding units, prediction units, and transformation units can all be used as image blocks obtained after division. Both the transformation unit and the transformation unit are divided based on the coding unit.

2. Video sampling

A pixel is the smallest complete sample of a video or image, so the data processing of an image block is performed in units of pixels. Among them, each pixel records color information, one represents color through RGB, R represents red (red), G represents green (green), B represents blue (blue); the other represents color through YUV, and Y represents brightness ( luminance), U represents the first chromaticity Cb, and V represents the second chromaticity Cr. Since people are more sensitive to luminance than chroma, the storage space can be reduced by storing more luminance and less chroma. In some embodiments, in video coding and decoding, video sampling is generally performed in YUV format, and the YUV format includes 420 sampling format, 444 sampling format, and the like. The sampling format determines the number of samples for two chroma based on the number of samples of luma. For example, assuming that a coding unit has 4×2 pixels, the format is as follows:

[Y0,U0,V0][Y1,U1,V1][Y2,U2,V2][Y3,U3,V3];

[Y4,U4,V4][Y5,U5,V5][Y6,U6,V6][Y7,U7,V7];

Among them, the 420 sampling format means that YUV is sampled in a 4:2:0 format, that is, the brightness and the first chroma or the second chroma are selected in a ratio of 4:2, where the first chroma and the second chroma are interlaced select. Then, the coding unit sampling selects the luminance Y0-Y3 of the first line, and the first chroma U0 and U2, and selects the luminance Y4-Y7 of the second line, and the second chroma V4 and V6. The coding unit is composed of a luma coding unit and a chrominance coding unit after sampling, wherein,

The luma coding unit is:

[Y0][Y1][Y2][Y3];

[Y4][Y5][Y6][Y7];

The chroma coding units are:

[U0][U2];

[V4][V6];

Similarly, the 444 sampling format means that YUV is sampled in a 4:4:4 format, that is, the brightness and the first chroma and the second chroma are selected in a ratio of 4:4:4.

Then the brightness coding unit sampled by the above coding unit is:

[Y0][Y1][Y2][Y3];

[Y4][Y5][Y6][Y7];

The chroma coding units are:

[U0,V0][U1,V1][U2,V2][U3,V3];

[U4,V4][U5,V5][U6,V6][U7,V7];

The above-mentioned luma coding unit and chroma coding unit obtained through sampling are used as data units for subsequent coding processing.

3. Forecast

The image block is predicted based on redundant characteristics such as temporal redundancy and spatial redundancy in the video sequence. For example, since there is a correlation between adjacent image blocks in the same image frame, the value of the current image block can be estimated (predicted) based on the adjacent image blocks. As another example, in different frames of images, for an image block at a certain coordinate position, the value of the image block at the previous moment has a correlation with the value of the image block at the next moment, so the value of the image block at the previous moment can be The value of predicts the value of the image block at the next moment. Wherein, the image block to be predicted is a prediction unit (PU). The manner in which the forecast is made is called the forecast mode. The prediction mode that performs prediction based on the correlation between adjacent image blocks in the same image frame is an intra-frame prediction mode. Wherein, the intra-frame prediction mode includes mean value (DC) prediction mode, angle prediction mode and planar (Planar) prediction mode.

In DC prediction mode, the average value of reference pixels is calculated as the predicted pixel. As shown in Figure 1a, the mean value of the reference pixels on the left and the upper side is calculated as the predicted pixel. Wherein, according to the width and height of the entire block, it is determined that reference pixels on the left side and/or upper side are used for calculation.

Angle prediction mode can also be called directional prediction mode, which predicts pixels in different directions based on reference pixels. Wherein, the angle prediction mode includes a horizontal prediction mode, a vertical prediction mode, a diagonal prediction mode and the like. In the vertical prediction mode shown in FIG. 1 b , the pixels in the column where the reference pixel is located are predicted based on the vertical downward direction of the reference pixel. When prediction modes in other directions are used for prediction, projections in different directions are calculated and combined with reference pixels to obtain predicted pixels.

The Planar prediction mode calculates the weight of the reference pixels in the horizontal row and vertical column to obtain the predicted pixel of the intersection point. As shown in Figure 1c, the predicted pixels on the right side of the horizontal row and the lower side of the vertical column are obtained according to the reference pixels on the left and upper sides, and then the predicted pixels are obtained according to the pixels at both ends of the horizontal row and vertical column.

In addition, the prediction mode also includes a block copy mode. The block copy mode is also called the block copy mode, which refers to finding the optimal reconstruction block from the decoded area of the current frame as the matching block, and using the reconstruction pixels of the matching block as the prediction pixels of the current block, as shown in Figure 1d.

The codec method provided in this application is applicable to a video codec system. Figure 2 shows the structure of the video codec system.

As shown in FIG. 2 , the video codec system includes a source device 20 and a destination device 21 . The source device 20 generates encoded video data. The source device 20 can also be called a video encoding device or video encoding device. The destination device 21 can decode the encoded video data generated by the source device 20. The destination device 21 also It may be called a video decoding device or a video decoding device. Source device 20 and/or destination device 21 may include at least one processor and a memory coupled to the at least one processor. The memory may include but not limited to Read-Only Memory (ROM), Random Access Memory (Random Access Memory, RAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM) ), flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, which is not specifically limited by the present application.

Source device 20 and destination device 21 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Electronic equipment such as televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

Destination device 21 may receive encoded video data from source device 20 via link 22 . Link 22 may include one or more media and/or devices capable of moving encoded video data from source device 20 to destination device 21 . In one example, link 22 may include one or more communication media that enable source device 20 to transmit encoded video data directly to destination device 21 in real-time. In this example, the source device 20 may modulate the encoded video data according to a communication standard (eg, a wireless communication protocol), and may transmit the modulated video data to the destination device 21 . The aforementioned one or more communication media may include wireless and/or wired communication media, for example: radio frequency (Radio Frequency, RF) spectrum, one or more physical transmission lines. The one or more communication media described above may form part of a packet-based network such as a local area network, a wide area network, or a global network (eg, the Internet), among others. The one or more communication media described above may include routers, switches, base stations, or other devices that enable communication from source device 20 to destination device 21 .

In another example, the encoded video data may be output from the output interface 203 to the storage device 23 . Similarly, encoded video data may be accessed from storage device 23 through input interface 213 . The storage device 23 may include a variety of local access data storage media, such as Blu-ray discs, high-density digital video discs (Digital Video Disc, DVD), read-only discs (Compact Disc Read-Only Memory, CD-ROM), flash Memory, or other suitable digital storage media for storing encoded video data.

In another example, storage device 23 may correspond to a file server or another intermediate storage device that stores encoded video data generated by source device 20 . In this example, destination device 21 may obtain its stored video data from storage device 23 via streaming or downloading. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to destination device 21 . For example, a file server may include a World Wide Web (Web) server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk driver.

Destination device 21 may access the encoded video data through any standard data connection, such as an Internet connection. Example types of data connections include wireless channels suitable for accessing encoded video data stored on a file server, wired connections (eg, cable modem, etc.), or a combination of both. The encoded video data can be transmitted from the file server by streaming, download transmission or a combination of both.

The decoding method of the present application is not limited to wireless application scenarios. Exemplarily, the coding and decoding method of the present application can be applied to video coding and decoding supporting the following multiple multimedia applications: air TV broadcasting, cable TV transmission, satellite TV transmission, streaming transmission Video transmission (eg, via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, a video codec system can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

It should be noted that the video codec system shown in FIG. 2 is only an example of a video codec system, and is not a limitation of the video codec system in this application. The codec method provided in this application is also applicable to a scenario where there is no data communication between the encoding device and the decoding device. In other examples, the video data to be encoded or the encoded video data may be retrieved from local storage, streamed over a network, or the like. The video encoding device can encode the video data to be encoded and store the encoded video data in the memory, and the video decoding device can also obtain the encoded video data from the memory and decode the encoded video data.

In FIG. 2 , a source device 20 includes a video source 201 , a video encoder 202 and an output interface 203 . In some examples, output interface 203 may include a modulator/demodulator (modem) and/or a transmitter. Video source 201 may include a video capture device (e.g., a video camera), a video archive containing previously captured video data, a video input interface to receive video data from a video content provider, and/or a computer graphics system, or a combination of such sources of video data.

Video encoder 202 may encode video data from video source 201 . In some examples, source device 20 transmits the encoded video data directly to destination device 21 via output interface 203 . In other examples, the encoded video data may also be stored on the storage device 23 for later access by the destination device 21 for decoding and/or playback.

In the example of FIG. 2 , the destination device 21 includes a display device 211 , a video decoder 212 and an input interface 213 . In some examples, input interface 213 includes a receiver and/or a modem. Input interface 213 may receive encoded video data via link 22 and/or from storage device 23 . The display device 211 may be integrated with the destination device 21 or may be external to the destination device 21 . In general, the display device 211 displays the decoded video data. The display device 211 may include various display devices, for example, a liquid crystal display, a plasma display, an organic light emitting diode display, or other types of display devices.

In some examples, video encoder 202 and video decoder 212 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to process Encoding of both audio and video in a common data stream or in separate data streams.

Video encoder 202 and video decoder 212 may include at least one microprocessor, digital signal processor (Digital Signal Processor, DSP), application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), field programmable gate array (Field Programmable Gate Array, FPGA), discrete logic, hardware, or any combination thereof. If the codec method provided in this application is implemented by software, the instructions for the software can be stored in a suitable non-volatile computer-readable storage medium, and at least one processor can be used to execute the instructions to implement the application. .

The video encoder 202 and the video decoder 212 in this application may operate according to a video compression standard (such as HEVC), or may also operate according to other industry standards, which is not specifically limited in this application.

Fig. 3 is a schematic block diagram of the video encoder 202 in the embodiment of the present application. The video encoder 202 can respectively perform prediction, transformation, quantization and entropy coding processes in the prediction module 31 , the transformation module 32 , the quantization module 33 and the entropy coding module 34 . The video encoder 202 also includes a segmentation module 30 and a summer 302 . For video block reconstruction, the video encoder 202 also includes an inverse quantization module 35 , an inverse transformation module 36 , a summer 301 and a reference image memory 37 .

As shown in FIG. 3 , video encoder 202 receives video data, and partition module 30 partitions the video data into raw blocks. This partitioning may also include partitioning into slices, blocks or other larger units, and video block partitioning, for example, according to Largest Coding Units (LCUs) and quadtree structures of coding units. Exemplarily, the video encoder 202 encodes in components of video blocks within the video slice to be encoded. In general, a slice may be divided into a number of original blocks (and possibly into a collection of original blocks called tiles). The sizes of the coding units CU, transform units PU and transform units TU are typically determined in the partitioning module 30 .

The prediction module 31 may provide the resulting intra-mode predicted or block copy mode predicted predicted block to the summer 302 to generate a residual block, and provide the predicted block to the summer 301 to be reconstructed to obtain a reconstructed block, which The reconstructed block is used as reference pixels for subsequent prediction. Wherein, the video encoder 202 forms a pixel difference value by subtracting the pixel value of the predicted block from the pixel value of the original block, and the pixel difference value is the residual block, and the data in the residual block may include brightness difference and chrominance difference . Summer 301 represents one or more components that perform this subtraction operation. The prediction module 31 can also send related syntax elements to the entropy coding module 34 for merging into the code stream.

The transform module 32 may divide the residual block into one or more transform units TU for transform. The transform module 32 transforms the residual block using DCT or DST to obtain transform coefficients. Transform module 32 may convert the residual block from the pixel domain to a transform domain (eg, the frequency domain). Currently commonly used transformation methods are DCT2 and DST7.

Transform module 32 may send the resulting transform coefficients to quantization module 33 . The quantization module 33 quantizes the transform coefficients to further reduce the code rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting quantization parameters. In some possible implementations, quantization module 33 may then perform a scan of the matrix comprising quantized transform coefficients. Alternatively, entropy encoding module 34 may perform a scan.

After quantization, entropy encoding module 34 may entropy encode the quantized transform coefficients. For example, the entropy coding module 34 can perform context-adaptive variable-length coding (Context-Adaptive Varialbe-Length Coding, CAVLC), context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmetic Coding, CABAC), syntax-based Context Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partitioning Entropy (PIPE) decoding or another entropy coding method or technique. Following entropy encoding by entropy encoding module 34 , the encoded codestream may be transmitted to video decoder 212 or archived for later transmission or retrieval by video decoder 212 .

The inverse quantization module 35 and the inverse transformation module 36 respectively apply inverse quantization and inverse transformation, and the summer 301 adds the inversely transformed residual block and the prediction block predicted by the prediction module 31 to generate a reconstruction block, which is used for subsequent original Reference pixels for block prediction. This reconstructed block is stored in the reference image memory 37 .

FIG. 4 is a schematic structural diagram of the video decoder 212 in the embodiment of the present application. As shown in FIG. 4 , the video decoder 212 includes an entropy decoding module 40 , a prediction module 41 , an inverse quantization module 42 , an inverse transformation module 43 , a summer 401 and a reference image memory 44 . In some possible implementations, video decoder 212 may perform an exemplary reciprocal decoding process to the encoding process described with respect to video encoder 202 from FIG. 3 .

During the decoding process, video decoder 212 receives a codestream of encoded video from video encoder 202 . The entropy decoding module 40 of the video decoder 212 entropy decodes the codestream to generate quantized coefficients and syntax elements. The entropy decoding module 40 forwards the syntax elements to the prediction module 41 . Video decoder 212 may receive the syntax elements at the video slice level and/or the video block level.

Inverse quantization module 42 inverse quantizes (eg, dequantizes) the quantized transform coefficients provided in the codestream and decoded by entropy decoding module 40 . The inverse quantization process may include determining the degree of quantization using quantization parameters calculated by video encoder 202 for each video block in the video slice, and likewise determining the degree of inverse quantization applied. The inverse transform module 43 applies inverse transform (for example, transform methods such as DCT and DST) to the inversely quantized transform coefficients, generates an inversely transformed residual block in the pixel domain according to the inversely transformed transform coefficients, In other words, the inversely quantized transform coefficients are then inversely transformed to generate residual coefficients. Wherein, the size of the inverse transformation unit is the same as that of the transformation unit, and the inverse transformation method and the transformation method adopt the corresponding forward transformation and inverse transformation in the same transformation method. For example, the inverse transformation of DCT and DST is inverse DCT, inverse DST or concept A similar inverse transformation process.

After prediction module 41 generates the prediction block, video decoder 212 forms a decoded video block by summing the inverse transformed residual block from inverse transformation module 43 with the prediction block. Summer 401 represents one or more components that perform this summation operation. A deblocking filter may also be applied to filter the decoded video blocks in order to remove blocking artifacts, if desired. The decoded video blocks in a given frame or picture are stored in reference picture memory 44 as reference pixels for subsequent predictions.

This application provides a possible implementation of video encoding/decoding, as shown in Figure 5. Figure 5 is a schematic flow diagram of a video encoding/decoding provided by this application. The implementation of video encoding/decoding includes process ① to process ⑤ , Process ① to Process ⑤ may be performed by any one or more of the above source device 20 , video encoder 202 , destination device 21 or video decoder 212 .

Process ①: Divide a frame of image into one or more non-overlapping parallel coding units. There is no dependency between the one or more parallel encoding units, and they can be completely parallel/independently encoded and decoded, as shown in FIG. 5 , the parallel encoding unit 1 and the parallel encoding unit 2 .

Process ②: For each parallel coding unit, it can be divided into one or more independent coding units that do not overlap with each other. Each independent coding unit can be independent of each other, but can share some parallel coding unit header information.

For example, an independent coding unit has a width of w_lcu and a height of h_lcu. If the parallel coding unit is divided into an independent coding unit, the size of the independent coding unit is exactly the same as that of the parallel coding unit; otherwise, the width of the independent coding unit should be larger than the height (except for the edge area).

Usually, the independent coding unit can be fixed w_lcu×h_lcu, both w_lcu and h_lcu are 2 to the Nth power (N≥0), for example, the size of the independent coding unit is: 128×4, 64×4, 32×4, 16×4, 8×4, 32×2, 16×2 or 8×2 etc.

As a possible example, the independent coding unit may be a fixed 128×4. If the size of the parallel coding unit is 256×8, the parallel coding unit can be divided into 4 independent coding units; if the size of the parallel coding unit is 288×10, the parallel coding unit can be divided into: the first/second row It includes two 128×4 independent coding units and one 32×4 independent coding unit; the third row includes two 128×2 independent coding units and one 32×2 independent coding unit.

It is worth noting that the independent coding unit may include three components of luma Y, first chroma Cb, and second chroma Cr, or three components of RGB, or only one of them. If the independent coding unit includes three components, the sizes of the three components may be exactly the same or different, which is specifically related to the input format of the image.

Process ③: For each independent coding unit, it can be divided into one or more non-overlapping coding units. Each coding unit in an independent coding unit can depend on each other. For example, multiple coding units can perform mutual reference precoding and decoding .

If the size of the coding unit is the same as that of the independent coding unit (that is, the independent coding unit is only divided into one coding unit), then its size can be all the sizes described in process ②.

If the independent coding unit is divided into multiple non-overlapping coding units, the feasible division examples are: horizontal equal division (the height of the coding unit is the same as that of the independent coding unit, but the width is different, and the width of the coding unit can be the width of the independent coding unit 1/2, 1/4, 1/8, 1/16, etc.), vertical equal division (the width of the coding unit is the same as that of the independent coding unit, but the height is different, and the height of the coding unit can be 1/ of the height of the

independent coding unit

2, 1/4, 1/8, 1/16, etc.), horizontal and vertical equal division (quadtree division), etc., preferably horizontal equal division.

The width of the coding unit is W and the height is H, so its width should be greater than its height (unless it is an edge area). Generally, the coding unit may be a fixed W×H, and both W and H are two Nth powers (N is greater than or equal to 0), such as 16x4, 8x4, 16x2, 8x2, 8x1, 4x1 and so on.

As a possible example, the coding unit may be fixed 16x4. If the size of the independent coding unit is 64x4, the independent coding unit can be equally divided into 4 coding units; if the size of the independent coding unit is 72x4, the coding unit can be divided into four 16x4 and one 8x4.

It should be noted that the coding unit may include three components of luma Y, first chroma Cb, and second chroma Cr (or RGB three components), or may only include one of them. If it contains three components, the sizes of several components can be exactly the same or different, depending on the image input format.

It is worth noting that the process ③ is an optional step in the video encoding and decoding method, and the video encoder/decoder can encode/decode the residual coefficient (or residual value) of the independent coding unit obtained in the process ②.

Process ④: For the coding unit, it can be divided into one or more non-overlapping prediction groups (Prediction Group, PG), PG can also be referred to as Group, and each prediction group is encoded and decoded according to the selected prediction mode , to obtain the prediction value of the prediction group to form the prediction value of the entire coding unit, and obtain the residual value of the coding unit based on the prediction value and the original value of the coding unit.

Process ⑤: Based on the residual value of the coding unit, the coding unit is grouped to obtain one or more non-overlapping residual blocks (residual block, RB), and the residual coefficients of each residual block are selected according to the selected mode Perform encoding and decoding to form a residual coefficient stream. In some embodiments, it can be divided into two categories: transforming and not transforming the residual coefficients.

Wherein, the selected mode of the residual coefficient encoding and decoding method in the process ⑤ may include, but not limited to any of the following: semi-fixed length encoding method, exponential Golomb (Golomb) encoding method, Golomb-Rice encoding method, truncated unary code Encoding methods, run-length encoding methods, direct encoding of raw residual values, etc.

For example, a video encoder may directly encode coefficients within a residual tile.

As another example, the video encoder may also perform transformation on the residual block, such as DCT, DST, Hadamard transformation, etc., and then encode the transformed coefficients.

As a possible example, when the small residual block is small, the video encoder may directly uniformly quantize the coefficients in the small residual block, and then perform binarization coding. If the residual small block is large, it can be further divided into multiple coefficient groups (coefficient group, CG), and then each coefficient group is uniformly quantized, and then binary coded. In some embodiments of the present application, the coefficient group (CG) and the quantization group (QG) may be the same.

The part of encoding the residual coefficients in a semi-fixed-length encoding manner will be exemplarily described below. First, the maximum value of the absolute value of the residual within a small residual block is defined as the modified maximum (MM). Secondly, the number of coding bits of the residual coefficients in the small residual block is determined (the number of coding bits of the residual coefficients in the same small residual block is the same). For example, if the critical limit (CL) of the current residual small block is 2 and the current residual coefficient is 1, then 2 bits are required to encode the residual coefficient 1, which is expressed as 01. If the key limit value of the current residual small block is 7, it means encoding 8-bit residual coefficient and 1-bit sign bit. The determination of the key limit is to find the minimum M value that satisfies all the residuals of the current sub-block within the range of [-2^(M-1), 2^(M-1)]. If there are two boundary values -2^(M-1) and 2^(M-1) at the same time, M should be increased by 1, that is, M+1 bits are required to encode all residuals of the current residual block; if only If there is one of the two boundary values -2^(M-1) and 2^(M-1), a Trailing bit needs to be encoded to determine whether the boundary value is -2^(M-1) or 2^(M -1); if none of -2^(M-1) and 2^(M-1) exists in all residuals, the Trailing bit does not need to be encoded.

In addition, for some special cases, the video encoder can also directly encode the original value of the image instead of the residual value.

The above video encoder 202 and video decoder 212 can also be implemented in another form of implementation, for example, by using a general-purpose digital processor system, such as the codec device 50 shown in Figure 6, the codec device 50 can be Some of the devices in the above video encoder 202 may also be some of the devices in the above video decoder 212 .

The codec device 50 can be applied to the encoding side or the decoding side. The codec device 50 includes a processor 501 and a memory 502 . The processor 501 is connected to the memory 502 (for example, connected to each other through a bus 504 ). In some examples, the codec device 50 may further include a communication interface 503 connected to the processor 501 and the memory 502 for receiving/sending data.

Memory 502 can be random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM) or portable read-only memory (Compact Disc Read-Only Memory, CD-ROM). The memory 502 is used to store related program codes and video data.

The processor 501 may be one or more central processing units (Central Processing Unit, CPU), such as CPU 0 and CPU 1 shown in FIG. 6 . In the case that the processor 501 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 501 is configured to read the program codes stored in the memory 502, so as to execute the operations of any one of the implementations corresponding to FIG. 7 to FIG. 11 and various feasible implementations thereof.

Hereinafter, the codec method provided by the present application will be described in detail with reference to the video codec system shown in FIG. 2 , the video encoder 202 shown in FIG. 3 , and the video decoder 212 shown in FIG. 4 .

As shown in FIG. 7 , it is a flow chart of a video decoding method provided by this application. The method includes step S601 to step S602.

S601. The video decoder performs entropy decoding and inverse quantization on the code stream to obtain an inverse transform unit.

S602. The video decoder performs discrete wavelet inverse transform on the inverse transform unit to obtain inversely transformed coefficients.

Discrete wavelet transform, the basic idea of discrete wavelet transform is translation and stretching. After high-pass filtering and low-pass filtering, the signal is decomposed in scale and space without losing the original information. The purpose of wavelet transform is to remove the correlation between pixels in each sub-image, and concentrate information on as few transform coefficients as possible, so that the next quantization step may quantize the coefficients carrying less information to 0, reducing the impact on the quality of the reconstructed image.

Among them, discrete wavelet transform includes discrete wavelet forward transform and discrete wavelet inverse transform. The video decoder determines the corresponding inverse transform method according to the transform method adopted by the video encoder.

The image block is a two-dimensional data array. At present, the two-dimensional discrete wavelet transform usually adopts the row and column separation processing method, that is, the horizontal direction and the vertical direction are respectively subjected to one-dimensional discrete wavelet transform. Wherein, the order of the horizontal direction and the vertical direction has no influence on the result.

In some embodiments, the video decoder performs inverse transformation in the horizontal direction and/or vertical direction, including seven inverse discrete wavelet transform combinations shown in Table 1 below. In some other embodiments, the video encoder performs transformation in the horizontal direction and/or vertical direction, to be combined similarly to 7 shown in Table 1 below, that is, to replace inverse discrete wavelet transform with discrete wavelet transform.

Table 1

组合combination	水平方向horizontal direction	垂直方向 vertical direction
11	离散小波反变换Inverse Discrete Wavelet Transform	不进行反变换no inverse transformation
22	不进行反变换no inverse transformation	离散小波反变换Inverse Discrete Wavelet Transform
33	离散小波反变换Inverse Discrete Wavelet Transform	DCT2反变换DCT2 inverse transform
44	DCT2反变换DCT2 inverse transform	离散小波反变换Inverse Discrete Wavelet Transform
55	离散小波反变换Inverse Discrete Wavelet Transform	DST7反变换DST7 inverse transform
66	DST7反变换DST7 inverse transform	离散小波反变换Inverse Discrete Wavelet Transform
77	离散小波反变换Inverse Discrete Wavelet Transform	离散小波反变换Inverse Discrete Wavelet Transform

Two discrete wavelet transforms are introduced below, namely Haar wavelet transform and 5/3 wavelet transform.

1. Haar wavelet transform

The Haar wavelet transform is a discrete wavelet transform with a low-pass filter of [1,1] and a high-pass filter of [-1,1]. The transformation steps are as follows: divide the data to be transformed into continuous pixel pairs, then calculate the average value of adjacent pixel pairs and half of the difference between adjacent pixel pairs, and place the calculated average value in the first half of the transformation coefficient, representing low-frequency DC Part, half of the difference between adjacent pixel pairs is placed in the second half of the transform coefficient, indicating the high-frequency detail part.

Suppose the original one-dimensional data a=[a ₁ ,a ₂ ,a ₃ ,a ₄ ], Haar low-pass filter [1,1], Haar high-pass filter [-1,1], then the Haar wavelet transform is:

That is to take two numbers from the array in turn, calculate their sum and difference, and save half of the sum and half of the difference in the first half and second half of the array in turn.

In some examples, the wavelet transform is performed on the low-frequency coefficient part, and the second-level wavelet decomposition is performed to obtain:

c ₂ , b ₃ , and b ₄ are called detail (high frequency) coefficients, and c ₁ is called a direct current (low frequency) coefficient. It is understandable that the higher the level of wavelet transform, the elements with little change in the data will approach zero, and the compression can be achieved with quantization.

The formula for doing Haar wavelet transform on a matrix X is Y=B·X· ^AT , where X is an MxN block and A and B are the Haar wavelet transform matrices of M points and N points respectively, and the Haar wavelet Inverse transformation is X=B ^T · Y · A.

The following is the Haar wavelet transformation matrix with M or N values of 2, 4 and 8:

In addition, when M or N= ^2k , the wavelet transform matrix is

where H except the 0th row

in,

There are N 1s in total, the second ^p + q line is h _{p, q} and satisfies:

p and q are integers greater than or equal to zero;

2. 5/3 wavelet transform

The high-pass filter coefficient of the 5/3 wavelet transform is Y(2n+1), and the low-pass filter coefficient is Y(2n).

Y(2n+1)=-X(2n)/2+X(2n+1)-X(2n+2)/2;

Y(2n)＝-X(2n-2)/8+X(2n-1)/4+3*X(2n)/4+X(2n+1)/4-X(2n+2)/8 +1/2;

In order to avoid the process of repeated calculation by directly using the above coefficients for transformation, Y(2n) can be calculated in the following way:

Y(2n)=X(2n)+(Y(2n-1)+Y(2n+1)+2)/4;

In some embodiments, by first calculating the high-pass filter coefficients, the low-pass filter coefficients are calculated according to the high-pass filter coefficients and corresponding source data.

As shown in Figure 8, assuming that the original sequence is x[0], x[1], ..., x[7], the number of data contained in the sequence is 8, and when n is 0, calculate the first low Y(2n-1) is needed when passing the filter coefficient, and in order to obtain Y(2n-1), it is necessary to expand a data on the left side of the high-pass filter coefficient, in order to obtain the high-pass filter coefficient expanded on the left side, it is necessary to expand in the original The left side of the sequence is extended by two data. Therefore, two data are extended on the left side of x[0], where x[-1]=x[1], x[-2]=x[2]. Similarly, when n is 3, Y(2n+1) needs to be used to calculate the low-pass filter coefficient, and in order to obtain Y(2n+1), it is necessary to extend a data on the right side of the original sequence. Therefore, one data is extended on the right side of x[7], x[8]=x[6]. Calculate the high-pass filter coefficient and the low-pass filter coefficient according to the extended x sequence.

As in the above example, X is a symmetrically expanded sample sequence. When the number of original data is even, according to the needs of the algorithm, two data are symmetrically expanded at the left end of the data, and one data is symmetrically expanded at the right end of the data sequence. When the number of original data is odd, two data should be expanded symmetrically on the left and right sides of the sequence, where, when expanding to the left, x[n]=x[n+2k], and when expanding to the right, x[n] =x[n-2k], k is an integer greater than zero, indicating the number of extensions.

It should be noted that the two discrete wavelet transform methods introduced above are applied to one-dimensional data. In two-dimensional data, transformations can be performed in the horizontal and vertical directions respectively, that is, calculations are performed based on rows and columns. Wherein, the transformation sequence of rows and columns does not affect the transformation coefficients.

By applying the above two discrete wavelet transform methods, the transform is completed through addition and shift calculations, reducing the computational complexity of video coding. Correspondingly, the burden of calculation and storage is reduced, and the complexity of hardware implementation is reduced.

The inverse discrete wavelet transform in combinations 1-7 in Table 1 above can be replaced by any one of inverse Haar wavelet transform and inverse 5/3 wavelet transform, as shown in Table 2 below. In some other embodiments, the video encoder performs transformation in the horizontal direction and/or vertical direction similar to the inverse transformation in Table 2, that is, the video encoder performs discrete wavelet transform in the horizontal direction and/or vertical direction, which can use H Any one of Err wavelet transform, 5/3 wavelet transform, DCT2 transform, DST7 transform or no transform.

Table 2

组合combination	水平方向horizontal direction	垂直方向 vertical direction
11	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform	不进行反变换no inverse transformation
22	不进行反变换no inverse transformation	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform
33	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform	DCT2反变换DCT2 inverse transform
44	DCT2反变换DCT2 inverse transform	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform
55	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform	DST7反变换DST7 inverse transform
66	DST7反变换DST7 inverse transform	哈尔小波反变换或5/3小波反变换Haar wavelet inverse transform or 5/3 wavelet inverse transform
77	哈尔小波反变换Inverse Haar wavelet transform	哈尔小波反变换Inverse Haar wavelet transform
88	5/3小波反变换5/3 wavelet inverse transform	5/3小波反变换5/3 wavelet inverse transform
99	哈尔小波反变换Inverse Haar wavelet transform	5/3小波反变换5/3 wavelet inverse transform
1010	5/3小波反变换5/3 wavelet inverse transform	哈尔小波反变换Inverse Haar wavelet transform

Among them, the combinations 1-6 in Table 2 all include two ways, therefore, the above Table 2 contains a total of 16 combinations of transformation methods.

In the above steps S601-S602, by adopting the discrete wavelet inverse transform, the computing resources occupied in the calculation process are reduced, and the calculation of multiplication is avoided through the translation and stretching according to the Haar wavelet inverse transform and the 5/3 wavelet inverse transform, It helps to reduce the computational burden when dealing with larger size data. On the basis of using discrete wavelet inverse transform, this application also proposes to choose one of horizontal direction and vertical direction for inverse transform, and by performing calculation in one dimension, the computational complexity is fundamentally reduced.

Corresponding to the decoding side shown in FIG. 7 , this embodiment of the present application provides an encoding method, which is applied in an encoding device, for example, in a device having a video encoding function. As shown in FIG. 9, the encoding method includes steps S801 to S802.

S801. The video encoder performs discrete wavelet transform on the transform unit to obtain transform coefficients.

Wherein, the video encoder performs discrete wavelet transform in the horizontal direction and/or vertical direction, including seven discrete wavelet transform combinations similar to those shown in Table 1 above.

In one example, discrete wavelet transform includes Haar wavelet transform and 5/3 wavelet transform. The video encoder performs discrete wavelet transform in the horizontal direction and/or vertical direction, including 16 discrete wavelet transform combinations similar to those shown in Table 2 above.

S802. The video encoder performs quantization and entropy encoding on the transform coefficients to obtain a code stream.

In the above steps S801-S802, by adopting the discrete wavelet transform, the computing resources occupied in the calculation process are reduced, and the calculation of multiplication is avoided through the translation and stretching according to the Haar wavelet transform and the 5/3 wavelet transform, which contributes to When dealing with larger size data, the computational burden can be reduced. On the basis of using discrete wavelet transform, this application also proposes to choose one of the horizontal direction and the vertical direction for transformation, and to fundamentally reduce the computational complexity by performing calculations in one dimension.

As shown in FIG. 10 , it is a flowchart of another video decoding method provided by this application. The method includes steps S901 to S902.

S901. The video decoder performs entropy decoding and inverse quantization on the code stream to obtain an inverse transform unit.

S902. The video decoder performs a third inverse transform on the inverse transform unit to obtain the inversely transformed coefficients; wherein, the third inverse transform includes inverse transform in the horizontal direction and/or inverse transform in the vertical direction, the horizontal direction and the vertical direction The inverse transformation methods include DCT2 inverse transformation or DST7 inverse transformation, and the inverse transformation methods in the horizontal direction and vertical direction are different.

In some embodiments, the video decoder performs the third inverse transformation of inverse transformation in the horizontal direction and/or vertical direction, including 8 kinds of inverse transformation combinations shown in Table 3 below:

table 3

组合combination	水平方向horizontal direction	垂直方向 vertical direction
11	DCT2反变换DCT2 inverse transform	不进行反变换no inverse transformation
22	不进行反变换no inverse transformation	DCT2反变换DCT2 inverse transform
33	DST7反变换DST7 inverse transform	不进行反变换no inverse transformation
44	不进行反变换no inverse transformation	DST7反变换DST7 inverse transform
55	DCT2反变换DCT2 inverse transform	DST7反变换DST7 inverse transform
66	DST7反变换DST7 inverse transform	DCT2反变换DCT2 inverse transform
77	DCT2反变换DCT2 inverse transform	DCT2反变换DCT2 inverse transform
88	DST7反变换DST7 inverse transform	DST7反变换DST7 inverse transform

Through the above steps S901-S902, different types of inverse transformation methods are implemented in the horizontal direction and vertical direction, and the flexibility of video decoding is improved. Among them, when only one direction of transformation is performed, it helps to reduce computational complexity and improve compression performance.

Corresponding to the decoding side shown in FIG. 10 , this embodiment of the present application provides an encoding method, which is applied in an encoding device, for example, in a device having a video encoding function. As shown in FIG. 11 , the encoding method includes steps S1001 to S1002.

S1001. The video encoder performs a third transformation on the transformation unit to obtain transformation coefficients; wherein, the third transformation includes transformation in the horizontal direction and/or transformation in the vertical direction, and the transformation methods in the horizontal direction and the vertical direction include DCT2 transformation or DST7 transform.

In some embodiments, the third transformation performed by the video decoder in the horizontal direction and/or vertical direction includes 8 transformation combinations similar to those shown in Table 3 above.

S1002. The video encoder performs quantization and entropy encoding on the transform coefficients to obtain a code stream.

Through the above steps S1001-S1007, different types of transformation methods are implemented in the horizontal direction and vertical direction, and the flexibility of video coding is improved. Among them, when only one direction of transformation is performed, it helps to reduce computational complexity and improve compression performance.

Based on the codec methods described above in FIG. 7 , FIG. 9 , FIG. 10 and FIG. 11 , the present application provides the following possible embodiments.

Embodiment 1. The video decoder determines the inverse transformation method based on the size of the inverse transformation unit.

Wherein, the size of the inverse transformation unit may be preset in the video decoder. Assume that the size of the inverse transform unit is M×N, where M represents the width and N represents the height. M×N can include 4×1, 8×1, 16×1, 4×2, 8×2, 16×2, 4×4, 8×4, 16×4, 8×8, 16×8, and 16 At least one of ×16, which is not limited in this application.

In an example, the size of the inverse transform unit is determined according to the size of the coding unit CU. Wherein, the size of the inverse transformation unit is the same as the size of the coding unit; or, the width of the inverse transformation unit is half of the width of the coding unit, and the height of the inverse transformation unit is the same as the height of the coding unit; or, the height of the inverse transformation unit is the coding unit half the height of the unit, and the width of the inverse transform unit is the same as the width of the coding unit. Exemplarily, if the size of the coding unit is W×H, where W is the width of the coding unit, and H is the height of the coding unit, then the size of the inverse transformation unit is w×h, where w is the width of the inverse transformation unit, and h is the height of the inverse transform unit. Wherein, w×h can be W×H, or w×h can be W/2×H, or w×h can be W×H/2, or w×h can be W/2×H/2. Among them, W×H can be 4×1, 8×1, 16×1, 4×2, 8×2, 16×2, 4×4, 8×4, 16×4, 8×8, 16×8 and any one of 16×16.

For example, the size of the coding unit is 16x4, and the size of the inverse transformation unit is 16x4; or, the size of the coding unit is 16x4, and the size of the inverse transformation unit is 8x4; or, the size of the coding unit is 16x2, and the size of the inverse transformation unit is 16x2; Alternatively, the size of the coding unit is 16x2, and the size of the inverse transform unit is 8x2.

It should be noted that, for the determination method of the size of the above coding unit, reference may be made to the process ② in FIG. 5 above. It can be understood that the inverse transformation unit is the smallest data unit for transformation, combined with the process ⑤ in Figure 5 above, the inverse transformation unit can be obtained by further dividing the residual block.

In an example, the size of the inverse transform unit is the same as the size of the prediction unit PU, and the size of the prediction unit is 4×1, 8×1, 16×1, 4×2, 8×2, 16×2, 4× Any one of 4, 8×4, 16×4, 8×8, 16×8 and 16×16.

The video decoder determines the inverse transform method according to the size of the inverse transform unit. When the inverse transform method includes discrete wavelet inverse transform, in some embodiments, the size of the inverse transform unit includes a first size and a second size, and based on the first size, it is determined to perform discrete wavelet inverse transform in the horizontal direction, and to perform discrete wavelet transform in the vertical direction. Inverse wavelet transform: based on the second dimension, discrete wavelet inverse transform is performed in the horizontal direction, and no transformation is performed in the vertical direction. Wherein, the first size and the second size may be any one of the above optional ways, and the inverse discrete wavelet transform includes any one of the inverse Haar wavelet transform or the inverse 5/3 wavelet transform.

It should be noted that the above correspondence between the first size, the second size and the inverse transformation method is only an example, and the discrete wavelet inverse transformation can also be performed in the vertical direction, and the inverse transformation is not performed in the horizontal direction.

When the transform method includes DCT2 transform and DST7 transform, similarly, when the inverse transform method includes DCT2 inverse transform and DST7 inverse transform, in some embodiments, the size of the inverse transform unit includes the third size, the fourth size, the fifth size size and sixth size. Determine the inverse transformation unit based on the third size to perform DCT2 inverse transformation or DST7 inverse transformation in the horizontal direction, and not perform inverse transformation in the vertical direction; determine the inverse transformation unit based on the fourth size without performing inverse transformation in the horizontal direction, and perform DCT2 inverse transformation in the vertical direction Transformation or DST7 inverse transformation; determine the inverse transformation unit based on the fifth size to perform DCT2 inverse transformation in the horizontal direction, and perform DST7 inverse transformation in the vertical direction; determine the inverse transformation unit based on the sixth size to perform DST7 inverse transformation in the horizontal direction, and perform DST7 inverse transformation in the vertical direction DCT2 inverse transform.

It should be noted that the inverse transform method can also include different combinations of discrete wavelet inverse transform, DCT2 inverse transform and DST7 inverse transform, for example, discrete wavelet inverse transform in the horizontal direction, DCT2 inverse transform or DST7 inverse transform in the vertical direction etc.

Through the above example, the video decoder determines the size of the inverse transformation unit, and determines the inverse transformation method based on the size of the inverse transformation unit, wherein the width and height of the size of the inverse transformation unit can correspond to different inverse transformation methods, which helps to reduce computational complexity to improve compression performance.

It should be noted that, on the encoding side, the manner of determining the transformation method according to the size of the transformation unit is the same as the manner of determining the inverse transformation method according to the size of the inverse transformation unit, which will not be repeated here.

It should be noted that when the size of the transformation unit is related to the size of the coding unit and the prediction unit, the size of the coding unit and the prediction unit is 4×1, 8×1, 16×1, 4×2, 8×2, 16 Any one of ×2, 4×4, 8×4, 16×4, 8×8, 16×8 and 16×16. The above-mentioned limitation on the size of the coding unit and the prediction unit, that is, the limitation on the height required by the hardware when line buffering is performed in the video coding and decoding technology, helps to reduce the complexity of hardware implementation.

Embodiment 2, the video decoder determines the inverse transformation method based on the transformation method identification bit in the code stream.

Wherein, the transformation method identification bit is used to indicate the transformation method of the transformation unit. For the video decoder, the transformation method is the method used by the video decoder to perform inverse transformation on the inverse transformation unit.

As shown in Table 4 below, it is an exemplary definition of the transformation method identification bit:

Table 4

It should be noted that the transformation method identification bits shown in Table 4 can represent several exemplary implementation methods of discrete wavelet transform in the horizontal direction and/or vertical direction, and also include discrete wavelet transform in the horizontal direction or vertical direction. For wavelet transform, DCT2 transform or DST7 transform is performed in another direction, and the identification bits of the multiple transformation methods can be increased or decreased by referring to the manner in Table 4, thereby expanding or reducing the number of transformation methods indicated by the identification bits. This application is not limited to this.

As shown in Table 5 below, it is another exemplary definition of the transformation method identification bit:

table 5

变换方法标识位transform method flag	0(00)0(00)	1(01)1(01)	2(10)2(10)	3(11)3(11)
水平方向horizontal direction	DCT2变换DCT2 transform	DST7变换DST7 transformation	\\	\\
垂直方向vertical direction	\\	\\	DCT2变换DCT2 transform	DST7变换DST7 transformation

It should be noted that the identification bits of the conversion method shown in Table 5 are only examples, and the identification bits can be increased or decreased, and the conversion methods corresponding to different identification bits are not limited in this application.

In an example, two transformation methods are preset in the video decoder, and the transformation method identification bits are used to identify the two transformation methods. As shown in Table 6 below, it is another exemplary definition of the transformation method identification bit:

Table 6

变换方法标识位transform method flag	0(0)0(0)	1(1)1(1)
水平方向 horizontal direction	5/3小波变换5/3 wavelet transform	哈尔小波变换Haar wavelet transform
垂直方向 vertical direction	5/3小波变换5/3 wavelet transform	哈尔小波变换Haar wavelet transform

Wherein, the transformation method identified by the transformation method identification bit may be determined by a video encoder based on a Rate-Distortion Optimized (RDO) cost in the video coding stage. The rate-distortion cost can be used to measure the performance of the transformation method, for example, it can be calculated by the following formula: J(mode)=D+λ×R; wherein, J(mode) represents the rate-distortion cost of different modes, D is the quantization distortion, It is represented by the mean square sum of the difference between the reconstructed video and the original image, λ is the Lagrangian multiplier, and R is the actual number of bits required for encoding in the current mode. As shown in FIG. 3 , part of the reconstructed block comes from the inversely quantized and inversely transformed residual block. Therefore, the reconstructed block can reflect the influence of the inversely transformed transformation method. The greater the rate-distortion cost, the worse the performance of the transformation method used in the encoding mode, and the video encoder can select a transformation method with better performance to determine the corresponding transformation method identification bit. For example, the two methods shown in Table 6 above perform transformation on transformation units of the same size, and it is determined through calculation that the rate-distortion cost of the 5/3 wavelet transformation method is relatively small, and the transformation method identification bit is 0.

In an example, after the calculation of the above-mentioned rate-distortion cost, the transformation method with better performance is represented by implicit derivation. Still taking the above-mentioned example as an example, the identification bit used to indicate the transform method with a lower rate-distortion cost is derived from the parity of the number of transform coefficients. Exemplarily, when the number of transformation coefficients is an even number, the transformation method identification bit is derived, and the transformation method identification bit includes 0 and 1, respectively used to indicate the combination of the two transformation methods shown in Table 6.

It should be noted that the method shown in Table 6 may also be any combination of any two of the above-mentioned Table 2 and Table 3, which is not limited in the present application.

Through the above example, the video decoder determines the transformation method based on the transformation method identification bit, wherein the transformation method identification bit includes a combination of any preset methods, or only identifies a transformation method with better performance. This identification method can It saves the identification bits occupied in the code stream and improves the compression performance at the same time.

It should be noted that, on the encoding side, the video encoder indicates the transformation method in the code stream with the transformation method identification bit, where the transformation method indicated by the transformation method identification bit is the same as the above-mentioned definite inverse transformation method on the decoding side, which is not mentioned here. Let me repeat.

Embodiment 3, the video decoder determines the inverse transformation method based on the prediction mode identification bit in the code stream.

Wherein, the prediction mode identification bit is used to indicate the prediction mode of the prediction unit for prediction before the inverse transformation unit performs transformation, wherein, the prediction mode includes DC prediction mode, angle prediction mode, Planar prediction mode and block copy mode, and the angle prediction mode includes horizontal prediction mode, vertical prediction mode and diagonal prediction mode.

As shown in Table 7 below, it is an exemplary definition of the prediction mode identification bit:

Table 7

预测模式标识位prediction mode flag	0(00)0(00)	1(01)1(01)	2(10)2(10)	3(11)3(11)
预测模式predictive model	垂直预测模式vertical prediction mode	DC预测模式DC prediction mode	Planar预测模式Planar prediction model	块拷贝模式block copy mode

It should be noted that there are multiple prediction modes, and only four are shown above as examples, including vertical prediction mode, DC prediction mode, Planar prediction mode and block copy mode in angle prediction modes. The prediction mode identification bit may also include more or less prediction modes, which is not limited in the present application.

In some embodiments, the video decoder determines the prediction mode according to the above-mentioned prediction mode identification bit, and at the same time determines the inverse transformation method corresponding to the prediction mode. As shown in Table 8 below, an exemplary correspondence between the prediction mode and the inverse transformation method:

Table 8

预测模式predictive model	垂直预测模式vertical prediction mode	DC预测模式DC prediction mode	Planar预测模式Planar prediction mode	块拷贝模式block copy mode
水平方向horizontal direction	哈尔小波反变换Inverse Haar wavelet transform	5/3小波反变换5/3 wavelet inverse transform	DCT2反变换DCT2 inverse transform	DST7反变换DST7 inverse transform
垂直方向 vertical direction	5/3小波反变换5/3 wavelet inverse transform	哈尔小波反变换Inverse Haar wavelet transform	\\	\\

In an example, when the prediction mode is the block copy mode, the predicted residual block is not subjected to inverse transformation, and the residual block is directly dequantized.

It should be noted that the correspondence between prediction modes and inverse transformation methods shown in Table 8 is only an example, and more or less correspondences may be included. Wherein, the corresponding relationship between the prediction mode and the inverse transformation method may be preset in the video decoder, or may be acquired through communication with the video encoder, which is not limited in the present application.

In an example, when the prediction mode has a corresponding relationship with the inverse transform method, it may also establish a corresponding relationship with the quantization method. For example, the inverse transformation method corresponding to the vertical prediction mode in Table 8 above is to perform inverse Haar wavelet transformation in the horizontal direction, and to perform 5/3 wavelet transformation inversion in the vertical direction, then the method used for quantization is scalar quantization. When the video decoder obtains the prediction mode identification bit, it can also determine the dequantization method.

Through the above example, the video decoder determines the inverse transformation method based on the prediction mode identification bit, wherein the prediction mode identification bit is used to identify one or more prediction modes, and the inverse transformation method is determined according to the prediction mode. This identification method helps to save code The identification bits occupied in the stream, and based on the residuals of different prediction modes, based on the performance of different transformation methods for residual processing, establish the optimal corresponding relationship, which helps to improve compression performance.

It should be noted that on the encoding side, the video encoder represents the prediction mode in the code stream with the prediction mode identification bit, where the prediction mode indicated by the prediction mode identification bit and its corresponding transformation method are the same as the above-mentioned determination method on the decoding side The same or similar, will not be repeated here.

Embodiment 4, the video decoder performs inverse transformation on the inverse transformation unit based on luma and chrominance.

In some embodiments, when the inverse transformation unit performs inverse transformation, calculations are performed based on the luma transformation unit and the chrominance transformation unit respectively. The luma transformation unit and the chroma transformation unit refer to the coding unit based on brightness and chroma obtained according to the sampling format, and the size of the inverse transformation unit is determined according to the size of the coding unit, wherein the inverse transformation unit includes a luma transformation unit and a chroma transformation unit . Exemplarily, as the example in the video sampling mentioned above, in the 420 sampling format, the luminance coding unit is:

[Y0][Y1][Y2][Y3];

[Y4][Y5][Y6][Y7];

The chroma coding units are:

[U0][U2];

[V4][V6];

When the size of the CU is the same as that of the inverse TU, the luma TU is the same as the luma CU, and the chroma TU is the same as the chroma CU.

The video decoder performs inverse transformation on the inverse transformation unit based on the luma transformation unit and the chroma transformation unit, including that the video decoder performs inverse transformation on the luma transformation unit in the horizontal direction and/or vertical direction, and performs inverse transformation on the chroma transformation unit in the horizontal direction and/or Or reverse transformation in the vertical direction. Wherein, the combination of inverse transformation methods in the horizontal direction and/or vertical direction may be any one in Table 2 and Table 3 above.

In one example, the video decoder determines the transformation method based on the sampling format. As shown in Table 9 below, an exemplary correspondence between the sampling format and the inverse transformation method:

Table 9

采样格式sampling format	444444	420420
亮度变换单元水平方向Brightness transformation unit horizontal direction	进行反变换inverse transformation	进行反变换inverse transformation
亮度变换单元垂直方向Brightness Transformation Unit Vertical Direction	进行反变换inverse transformation	进行反变换inverse transformation
色度变换单元水平方向Chroma Transformation Unit Horizontal Direction	进行反变换inverse transformation	进行反变换inverse transformation
色度变换单元垂直方向Chroma Transformation Unit Vertical Direction	进行反变换inverse transformation	不进行反变换no inverse transformation

As shown in Table 9, when the 444 sampling format is used, the inverse transformation is performed on both the luminance transformation unit and the chroma transformation unit in the horizontal and vertical directions; Inverse transformation is performed, and the chroma transformation unit is only inversely transformed in the horizontal direction. Wherein, the above-mentioned transformation method adopted for the luma transformation unit and the chroma transformation unit may be any combination in Table 2 and Table 3 above.

In one example, the video decoder determines the transform method based on the size of the luma transform unit and the chroma transform unit. As shown in Table 10 below, an exemplary correspondence between the size of the luma transformation unit and the chroma transformation unit and the transformation method:

Table 10

亮度变换单元的尺寸The size of the luma transform unit	16×216×2	16×216×2
色度变换单元的尺寸The size of the chroma transform unit	16×216×2	8×18×1
亮度变换单元水平方向Brightness transformation unit horizontal direction	进行反变换inverse transformation	进行反变换inverse transformation
亮度变换单元垂直方向Brightness Transformation Unit Vertical Direction	进行反变换inverse transformation	进行反变换inverse transformation
色度变换单元水平方向Chroma Transformation Unit Horizontal Direction	进行反变换inverse transformation	进行反变换inverse transformation
色度变换单元垂直方向Chroma Transformation Unit Vertical Direction	进行反变换inverse transformation	不进行反变换no inverse transformation

As shown in Table 10, the size of the luma transform unit is the same as that of the chroma transform unit, and inverse transformation is performed on both the luma transform unit and the chroma transform unit in the horizontal direction and the vertical direction; or, the luma transform unit is transformed in the horizontal direction and The inverse transformation is performed in the vertical direction. When the height of the chroma transformation unit is 1, the inverse transformation is only performed on the chroma transformation unit in the horizontal direction. Wherein, the above-mentioned transformation method adopted for the luma transformation unit and the chroma transformation unit may be any combination in Table 2 and Table 3 above.

It should be noted that the above Table 9 and Table 10 are only examples. Different sampling formats can correspond to different inverse transformation methods or reciprocal transformation methods. Different sizes of luma transformation units and chroma transformation units correspond to different inverse transformation methods. The transformation method or the reciprocal transformation method is not limited in this application.

It should be noted that on the encoding side, the video encoder transforms the transform unit based on luma and chroma, where the luma transform unit and the chroma transform unit are determined during sampling, and the horizontal and/or vertical directions are respectively The method of performing transformation by the luma transformation unit and the chrominance transformation unit is the same as or similar to the above-mentioned method on the decoding side, and will not be repeated here.

Embodiment 5, the video decoder decomposes the number of layers based on the discrete wavelet transform in the code stream.

In some embodiments, the code stream including image data obtained by the video decoder includes a sequence parameter set (Sequence Parameter Set, SPS), a picture parameter set (Picture Parameter Set, PPS) and a slice header (slice header) or a slice segment Header (slice segment header) and other syntax elements. Wherein, the sequence parameter set includes discrete wavelet transform decomposition layers. The video decoder performs discrete wavelet transform according to discrete wavelet transform decomposition layers. For example, if the number of discrete wavelet transform decomposition layers in the vertical direction is L, then the inverse transform unit performs vertical wavelet decomposition of L layers in the vertical direction. Wherein, the height of the inverse transformation unit is N, and 2L≤N. The number of discrete wavelet transform decomposition layers is suitable for Haar wavelet transform and 5/3 wavelet transform.

It should be noted that on the encoding side, the video encoder transforms according to the preset discrete wavelet transform decomposition layers, writes the decomposition layers into the sequence parameter set, and sends them to the video decoder for decoding. The discrete wavelet transform decomposition layers For related descriptions, please refer to the above, and will not repeat them here.

It should be noted that the above embodiment can be applied in steps S601-S602, S801-S802, S901-S902 and S1001-S1002. The specific implementation manner for the video encoder to perform transformation based on the transformation unit is a reciprocal coding process with the above-mentioned embodiment on the decoding side, and the description will not be repeated.

An embodiment of the present application provides a decoding device, and the decoding device may be a video decoder. The decoding device is used to perform the steps performed by the video decoder in the above decoding method. The decoding device provided in the embodiment of the present application may include modules corresponding to corresponding steps.

In the embodiment of the present application, the functional modules of the decoding device may be divided according to the above method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. The division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 12 shows a possible structural diagram of the decoding device involved in the above embodiment. As shown in FIG. 12 , the decoding device 110 includes a decoding module 1101 , an inverse transformation module 1102 and a storage module 1103 .

The decoding module 1101 is configured to perform entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit, such as the above step S601.

The inverse transform module 1102 is configured to perform discrete wavelet inverse transform on the inverse transform unit to obtain inversely transformed coefficients, such as the above step S602.

In one example, the inverse transform module 1102 is configured to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and perform discrete wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction or, perform any one of DCT2 inverse transform, DST7 inverse transform, and no inverse transform on the inverse transform unit in the horizontal direction, and perform discrete wavelet inverse transform in the vertical direction.

In one example, the inverse discrete wavelet transform includes inverse Haar wavelet transform and inverse 5/3 wavelet transform, and the inverse transform module 1102 is used to perform inverse Haar wavelet transform on the inverse transform unit in the horizontal direction, and in the vertical direction Perform any one of Haar wavelet inverse transformation, 5/3 wavelet inverse transformation, DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation; or, perform 5/3 wavelet inverse transformation on the inverse transformation unit in the horizontal direction , performing any one of Haar wavelet inverse transformation, 5/3 wavelet inverse transformation, DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation in the vertical direction; or, performing DCT2 inverse transformation on the inverse transformation unit in the horizontal direction transformation, DST7 inverse transformation and no inverse transformation, carry out Haar wavelet inverse transformation in the vertical direction; or, perform DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation on the inverse transformation unit in the horizontal direction In any one of them, the 5/3 wavelet inverse transform is performed in the vertical direction.

In an example, the size of the inverse transformation unit is the same as the size of the coding unit; or, the width of the inverse transformation unit is half of the width of the coding unit, and the height of the inverse transformation unit is the same as the height of the coding unit ; or, the height of the inverse transformation unit is half of the height of the coding unit, and the width of the inverse transformation unit is the same as the width of the coding unit.

In an example, the inverse transformation module 1102 is configured to determine, based on the size of the inverse transformation unit, to perform the first inverse transformation on the inverse transformation unit in the horizontal direction, and to perform the second inverse transformation in the vertical direction; wherein, the first transformation and at least one of the second transform is an inverse discrete wavelet transform.

In one example, the size of the inverse transform unit includes a first size and a second size, and the inverse transform module 1102 is configured to determine to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction and vertically Inverse discrete wavelet transform is performed in the direction; based on the second size, it is determined that the inverse transform unit performs inverse discrete wavelet transform in the horizontal direction, and does not perform inverse transform in the vertical direction.

In one example, the decoding module 1101 is configured to obtain the transformation method identification bit in the code stream, the transformation method identification bit is used to indicate the inverse transformation method of the inverse transformation unit, and the inverse transformation method includes and/or inverse discrete wavelet transform in the vertical direction.

In an example, the decoding module 1101 is configured to obtain a prediction mode identification bit in the code stream, and the prediction mode identification bit is used to indicate the prediction mode of the prediction unit for prediction before the inverse transformation unit performs inverse transformation , the prediction mode includes a first prediction mode and a second prediction mode; based on the first prediction mode, it is determined to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and perform discrete wavelet inverse transform, DCT2 inverse transform, DST7 in the vertical direction Any one of inverse transformation and no inverse transformation; based on the second prediction mode, it is determined to perform any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation on the inverse transformation unit in the horizontal direction. Inverse discrete wavelet transform for direction.

In one example, the prediction mode includes any one of DC prediction mode, angle prediction mode, Planar prediction mode and block copy mode, and the angle prediction mode is any one of horizontal prediction mode, vertical prediction mode and diagonal prediction mode kind.

In one example, the inverse transform module 1102 is configured to perform discrete wavelet inverse transform on the luminance of the inverse transform unit, and perform discrete wavelet inverse transform, DCT2 inverse transform and DST7 inverse transform on the chroma of the inverse transform unit or, perform any one of DCT2 inverse transform and DST7 inverse transform on the luminance of the inverse transform unit, and perform discrete wavelet inverse transform on the chrominance of the inverse transform unit.

Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here.

Of course, the decoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the decoding device 110 may further include a storage module 1103 .

The storage module 1103 can be used to store program codes and data of the decoding device.

In the case of dividing each functional module corresponding to each function, Fig. 13 shows a possible structural diagram of the decoding device involved in the above embodiment. As shown in FIG. 13 , the decoding device 120 includes a decoding module 1201 , an inverse transformation module 1202 and a storage module 1203 .

The decoding module 1201 is configured to perform entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit. For example, step S901 above.

The inverse transformation module 1202 is configured to perform a third inverse transformation on the inverse transformation unit to obtain inversely transformed coefficients; wherein, the third inverse transformation includes inverse transformation in the horizontal direction and/or inverse transformation in the vertical direction, horizontal direction and The inverse transformation method in the vertical direction includes DCT2 inverse transformation or DST7 inverse transformation. For example, step S902 above.

In an example, the size of the inverse transformation unit includes a third size, a fourth size, a fifth size, and a sixth size, and the inverse transformation module 1202 is configured to, based on the third size of the inverse transformation unit, The transformation unit performs DCT2 inverse transformation or DST7 inverse transformation in the horizontal direction, and does not perform inverse transformation in the vertical direction; based on the fourth size of the inverse transformation unit, the inverse transformation unit does not perform inverse transformation in the horizontal direction, and performs DCT2 inverse transformation in the vertical direction Or DST7 inverse transformation; based on the fifth size of the inverse transformation unit, DCT2 inverse transformation is performed on the inverse transformation unit in the horizontal direction, and DST7 inverse transformation is performed on the vertical direction; based on the sixth size of the inverse transformation unit, the inverse transformation unit is horizontally transformed Perform DST7 inverse transformation, and perform DCT2 inverse transformation in the vertical direction.

In an example, the decoding module 1201 is configured to obtain the transformation method identification bit in the code stream, and the transformation method identification bit is used to indicate the inverse transformation method of the inverse transformation unit; determine according to the inverse transformation method of the inverse transformation unit The third inverse transformation.

In one example, the decoding module 1201 is configured to obtain the prediction mode identification bit in the code stream, and the prediction mode identification bit is used to indicate the prediction mode for the prediction unit to perform prediction before the inverse transformation unit performs inverse transformation, and the prediction mode includes the third Prediction mode, fourth prediction mode, fifth prediction mode, and sixth prediction mode; based on the third prediction mode, it is determined to perform DCT2 inverse transformation or DST7 inverse transformation on the inverse transformation unit in the horizontal direction, and not perform inverse transformation in the vertical direction ;Based on the fourth prediction mode, it is determined that the inverse transformation unit is not inversely transformed in the horizontal direction, and DCT2 inverse transformation or DST7 inverse transformation is performed in the vertical direction; based on the fifth prediction mode, DCT2 inverse transformation is performed on the inverse transformation unit in the horizontal direction For transformation, DST7 inverse transformation is performed in the vertical direction; based on the sixth prediction mode, DST7 inverse transformation is performed on the inverse transformation unit in the horizontal direction, and DCT2 inverse transformation is performed in the vertical direction.

In an example, the prediction mode includes any one of DC mode, angle mode, Planar mode and block copy mode, and the angle mode is any one of horizontal mode, vertical mode and diagonal mode.

In one example, the inverse transform module 1202 is configured to perform DCT2 inverse transform or DST7 inverse transform on the luminance of the inverse transform unit; perform DCT2 inverse transform or DST7 inverse transform on the chroma of the inverse transform unit to obtain inversely transformed coefficients .

Of course, the decoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the decoding device may further include a storage module 1203 .

The storage module 1203 can be used to store program codes and data of the decoding device.

As an example, with reference to FIG. 6 , some or all of the functions implemented in the decoding module, the inverse transformation module, and the storage module in the decoding device 110 and the decoding device 120 can be executed by the processor 501 in FIG. 6 in the memory 502 in FIG. 6 . program code implementation.

In the case of dividing each functional module corresponding to each function, FIG. 14 shows a possible structural schematic diagram of the encoding device involved in the above embodiment. As shown in FIG. 14 , the encoding device 130 includes a transformation module 1301 , an encoding module 1302 and a storage module 1303 .

The transformation module 1301 is configured to perform discrete wavelet transformation on the transformation unit to obtain transformation coefficients. For example, step S801 above.

The encoding module 1302 is configured to perform quantization and entropy encoding on the transform coefficients to obtain a code stream. For example, step S802 above.

Of course, the encoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the encoding device may further include a storage module 1303 .

The storage module 1303 can be used to store program codes and data of the encoding device.

As an example, with reference to FIG. 6, some or all of the functions implemented in the transformation module 1301, the encoding module 1302, and the storage module 1303 in the encoding device 130 can be executed by the processor 501 in FIG. 6. The program in the memory 502 in FIG. 6 Code.

In the case of dividing each functional module corresponding to each function, FIG. 15 shows a possible structural diagram of the encoding device involved in the above embodiment. As shown in FIG. 15 , the encoding device 140 includes a transformation module 1401 , an encoding module 1402 and a storage module 1403 .

The transformation module 1401 is configured to perform a third transformation on the transformation unit to obtain transformation coefficients; wherein, the third transformation includes performing transformation in the horizontal direction and/or performing transformation in the vertical direction, and the transformation methods in the horizontal direction and vertical direction include DCT2 transformation or DST7 transformation. For example, the above step S1001.

The encoding module 1402 is configured to perform quantization and entropy encoding on the transform coefficients to obtain a code stream. For example, the above step S1002.

Of course, the encoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the encoding device may further include a storage module 1403 .

The storage module 1403 can be used to store program codes and data of the encoding device.

As an example, with reference to FIG. 6 , some or all of the functions implemented in the transformation module 1401, the encoding module 1402, and the storage module 1403 in the encoding device 140 can be executed by the processor 501 in FIG. 6 in the program in the memory 502 in FIG. 6 Code.

The embodiment of the present application also provides an electronic device, the electronic device includes the decoding device 110 and the decoding device 120 described above, and the decoding device 110 and the decoding device 120 execute any method performed by the video decoder provided above.

The embodiment of the present application also provides an electronic device, the electronic device includes the encoding device 130 and the encoding device 140 described above, and the encoding device 130 and the encoding device 140 execute any method performed by the video encoder provided above.

The embodiment of the present application also provides an electronic device, which includes the above-mentioned decoding device 110 and encoding device 130, and the decoding device 110 and encoding device 130 execute any video decoder and video encoder provided above. Methods.

The embodiment of the present application also provides an electronic device, which includes the above-mentioned decoding device 120 and encoding device 140, and the decoding device 120 and encoding device 140 execute any video decoder and video encoder provided above. Methods.

The embodiment of the present application also provides a communication system, which includes electronic equipment composed of the decoding device 110, the decoding device 120, the encoding device 130 and the encoding device 140, the decoding device 110, the decoding device 120, and the encoding device 130 And the encoding device 140 implements any one of the methods performed by the video decoder and video encoder provided above.

The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run on a computer, the computer is made to execute any video decoding method provided above. The method executed by the device.

Regarding the explanation of relevant content and the description of beneficial effects in any computer-readable storage medium provided above, reference may be made to the above-mentioned corresponding embodiments, and details are not repeated here.

The embodiment of the present application also provides a processor. The processor integrates a control circuit and one or more ports for realizing the functions of the above-mentioned decoding device 110 , decoding device 120 , encoding device 130 and encoding device 140 . Optionally, the functions supported by the processor can be referred to above, and will not be repeated here. Those of ordinary skill in the art can understand that all or part of the steps for implementing the above-mentioned embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium mentioned above may be a read-only memory, a random access memory, and the like. The above-mentioned processing unit or processor can be a central processing unit, a general-purpose processor, a specific integrated circuit (application specific integrated circuit, ASIC), a microprocessor (digital signal processor, DSP), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

The embodiments of the present application also provide a computer program product containing instructions, which, when the instructions are run on a computer, cause the computer to execute any one of the methods in the foregoing embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or may contain one or more data storage devices such as servers and data centers that can be integrated with the medium. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, SSD), etc.

It should be noted that the above-mentioned devices for storing computer instructions or computer programs provided by the embodiments of the present application, such as but not limited to, the above-mentioned memory, computer-readable storage medium, and communication equipment, etc., all have non-transitory .

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or can contain one or more data storage devices such as servers and data centers that can be integrated with the medium. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (solid state disk, SSD)), etc.

Although the present application has been described in conjunction with various embodiments herein, those skilled in the art can understand and realize the disclosure by viewing the drawings, the disclosure, and the appended claims during the implementation of the claimed application. Other Variations of Embodiments. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely illustrative of the application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

A video decoding method, comprising:

Perform entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit;

Inverse discrete wavelet transform is performed on the inverse transform unit to obtain coefficients after inverse transform.
The method according to claim 1, wherein, performing discrete wavelet inverse transform on the inverse transform unit to obtain coefficients after inverse transform, comprising:

Perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and perform any one of discrete wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction; or,

The inverse transformation unit performs any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation in the horizontal direction, and performs discrete wavelet inverse transformation in the vertical direction.
The method according to claim 2, wherein said discrete wavelet inverse transform comprises Haar wavelet inverse transform and 5/3 wavelet inverse transform,

Carrying out discrete wavelet inverse transform to described inverse transform unit, obtains the coefficient after inverse transform, comprises:

Perform inverse Haar wavelet transform on the inverse transform unit in the horizontal direction, inverse Haar wavelet transform, 5/3 wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction any of the following; or,

Carry out 5/3 wavelet inverse transform to described inverse transform unit in described horizontal direction, carry out Haar wavelet inverse transform, 5/3 wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and do not carry out inverse transform in described vertical direction any of the following; or,

Perform any one of DCT2 inverse transformation, DST7 inverse transformation and no inverse transformation on the inverse transformation unit in the horizontal direction, and perform Haar wavelet inverse transformation in the vertical direction; or,

Perform any one of DCT2 inverse transform, DST7 inverse transform and no inverse transform in the horizontal direction on the inverse transform unit, and perform 5/3 wavelet inverse transform in the vertical direction.
The method according to claim 1, wherein,

The size of the inverse transform unit is the same as the size of the coding unit; or,

The width of the inverse transformation unit is half the width of the coding unit, and the height of the inverse transformation unit is the same as the height of the coding unit; or,

The height of the inverse transform unit is half the height of the coding unit, and the width of the inverse transform unit is the same as the width of the coding unit.
The method according to claim 1, wherein, performing discrete wavelet inverse transform on the inverse transform unit to obtain coefficients after inverse transform, comprising:

Based on the size of the inverse transformation unit, it is determined to perform the first inverse transformation on the inverse transformation unit in the horizontal direction, and perform the second inverse transformation in the vertical direction; wherein, in the first inverse transformation and the second inverse transformation At least one of is discrete wavelet inverse transform.
The method according to claim 5, wherein the size of the inverse transform unit comprises a first size and a second size,

Based on the size of the inverse transformation unit, it is determined to perform the first inverse transformation on the inverse transformation unit in the horizontal direction, and perform the second inverse transformation in the vertical direction, including:

Based on the first size, determine to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and perform discrete wavelet inverse transform in the vertical direction;

Based on the second size, it is determined to perform inverse discrete wavelet transform on the inverse transform unit in the horizontal direction, and not to perform inverse transform in the vertical direction.
The method according to claim 1, wherein performing entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit comprises:

Obtain the transformation method identification bit in the code stream, the transformation method identification bit is used to indicate the inverse transformation method of the inverse transformation unit, and the inverse transformation method includes performing the horizontal and/or vertical transformation of the inverse transformation unit Inverse discrete wavelet transform for direction.
The method according to claim 1, wherein performing entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit comprises:

Obtain the prediction mode identification bit in the code stream, the prediction mode identification bit is used to indicate the prediction mode in which the prediction unit performs prediction before the inverse transformation unit performs inverse transformation, and the prediction mode includes the first prediction mode and the second prediction mode Two prediction models;

Based on the first prediction mode, it is determined to perform discrete wavelet inverse transform on the inverse transform unit in the horizontal direction, and perform any one of discrete wavelet inverse transform, DCT2 inverse transform, DST7 inverse transform and no inverse transform in the vertical direction ;

Based on the second prediction mode, it is determined to perform any one of DCT2 inverse transform, DST7 inverse transform, and no inverse transform in the horizontal direction on the inverse transform unit, and perform discrete wavelet inverse transform in the vertical direction.
The method according to claim 8, wherein the prediction mode includes any one of DC prediction mode, angle prediction mode, Planar prediction mode and block copy mode, and the angle prediction mode includes horizontal prediction mode, vertical prediction mode and any of the diagonal prediction modes.
The method according to claim 1, wherein, performing discrete wavelet inverse transform on the inverse transform unit to obtain coefficients after inverse transform, comprising:

Perform discrete wavelet inverse transform on the brightness of the inverse transform unit, and perform any one of discrete wavelet inverse transform, DCT2 inverse transform and DST7 inverse transform on the chroma of the inverse transform unit; or,

Perform any one of DCT2 inverse transform and DST7 inverse transform on the luminance of the inverse transform unit, and perform discrete wavelet inverse transform on the chroma of the inverse transform unit.
A video decoding method, comprising:

Perform entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit;

Performing a third inverse transform on the inverse transform unit to obtain inversely transformed coefficients; wherein the third inverse transform includes performing inverse transform on the inverse transform unit in the horizontal direction and/or performing inverse transform in the vertical direction, The inverse transformation method of the horizontal direction and the vertical direction includes DCT2 inverse transformation or DST7 inverse transformation.
The method of claim 11, wherein,

The size of the inverse transform unit is the same as the size of the coding unit; or,

The width of the inverse transformation unit is half the width of the coding unit, and the height of the inverse transformation unit is the same as the height of the coding unit; or,

The height of the inverse transform unit is half the height of the coding unit, and the width of the inverse transform unit is the same as the width of the coding unit.
The method according to claim 12, wherein the size of the inverse transform unit comprises a third size, a fourth size, a fifth size and a sixth size,

Performing a third inverse transform on the inverse transform unit to obtain inversely transformed coefficients, including:

Based on the third size of the inverse transformation unit, perform DCT2 inverse transformation or DST7 inverse transformation on the inverse transformation unit in the horizontal direction, and do not perform inverse transformation in the vertical direction;

Based on the fourth size of the inverse transformation unit, the inverse transformation unit is not subjected to inverse transformation in the horizontal direction, and DCT2 inverse transformation or DST7 inverse transformation is performed in the vertical direction;

Based on the fifth size of the inverse transformation unit, perform DCT2 inverse transformation on the inverse transformation unit in the horizontal direction, and perform DST7 inverse transformation in the vertical direction;

Based on the sixth size of the inverse transform unit, perform DST7 inverse transform in the horizontal direction and perform DCT2 inverse transform in the vertical direction on the inverse transform unit.
The method according to claim 11, wherein performing entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit comprises:

Acquiring a transformation method identification bit in the code stream, where the transformation method identification bit is used to indicate the inverse transformation method of the inverse transformation unit;

The third inverse transform is determined according to an inverse transform method of the inverse transform unit.
The method according to claim 11, wherein performing entropy decoding and inverse quantization on the code stream to obtain an inverse transformation unit comprises:

Obtain the prediction mode identification bit in the code stream, the prediction mode identification bit is used to indicate the prediction mode of the prediction unit for prediction before the inverse transformation unit performs inverse transformation, and the prediction mode includes the third prediction mode, the fourth prediction mode mode, a fifth predictive mode and a sixth predictive mode;

Based on the third prediction mode, determine to perform DCT2 inverse transformation or DST7 inverse transformation on the inverse transformation unit in the horizontal direction, and not perform inverse transformation in the vertical direction;

Based on the fourth prediction mode, it is determined not to perform inverse transformation on the inverse transformation unit in the horizontal direction, and to perform DCT2 inverse transformation or DST7 inverse transformation in the vertical direction;

Based on the fifth prediction mode, perform DCT2 inverse transformation on the inverse transformation unit in the horizontal direction, and perform DST7 inverse transformation in the vertical direction;

Based on the sixth prediction mode, DST7 inverse transform is performed on the inverse transform unit in the horizontal direction, and DCT2 inverse transform is performed in the vertical direction.
The method according to claim 15, wherein the prediction mode comprises any one of DC mode, angle mode, Planar mode and block copy mode, and the angle mode is one of horizontal mode, vertical mode and diagonal mode any kind.
The method according to claim 11, wherein performing the third inverse transformation on the inverse transformation unit to obtain the coefficients after inverse transformation comprises:

Perform DCT2 inverse transformation or DST7 inverse transformation on the brightness of the inverse transformation unit;

Perform DCT2 inverse transform or DST7 inverse transform on the chroma of the inverse transform unit to obtain inversely transformed coefficients.
A video coding method, comprising:

Perform discrete wavelet transform on the transform unit to obtain transform coefficients;

Perform quantization and entropy coding on the transform coefficients to obtain a code stream.
A video coding method, comprising:

performing a third transformation on the transformation unit to obtain transformation coefficients; wherein, the third transformation includes transformation in the horizontal direction and/or transformation in the vertical direction, and the transformation methods in the horizontal direction and the vertical direction include DCT2 transformation or dst7 transform;

Perform quantization and entropy coding on the transform coefficients to obtain a code stream.
A video decoding device, comprising:

The decoding module is used to perform entropy encoding and inverse quantization on the code stream to obtain an inverse transformation unit;

The inverse transform module is configured to perform discrete wavelet inverse transform on the inverse transform unit to obtain inversely transformed coefficients.
A video decoding device, comprising:

The decoding module is used to perform entropy encoding and inverse quantization on the code stream to obtain an inverse transformation unit;

An inverse transform module, performing a third inverse transform on the inverse transform unit to obtain inversely transformed coefficients; wherein, the third inverse transform includes transforming in the horizontal direction and/or performing inverse transform in the vertical direction, and the horizontal The transformation methods of the direction and the vertical direction include DCT2 inverse transformation or DST7 inverse transformation.
A video encoding device, comprising:

Transformation module, is used for carrying out discrete wavelet transformation to transformation unit, obtains transformation coefficient;

The coding module is used to perform quantization and entropy coding on the transformation coefficients to obtain code streams.
A video encoding device, comprising:

A transformation module, configured to perform a third transformation on the transformation unit to obtain transformation coefficients; wherein, the third transformation includes transformation in the horizontal direction and/or transformation in the vertical direction, and the transformation in the horizontal direction and the vertical direction The method includes DCT2 transformation or DST7 transformation;

The coding module is used to perform quantization and entropy coding on the transformation coefficients to obtain code streams.
A video decoder, configured to execute the method according to any one of claims 1-17.
A video encoder for performing the method as claimed in claim 18 or 19.
An electronic device, comprising a video decoder according to claim 24, a memory and a communication interface, the video decoder performing the method according to any one of the above claims 1-10, and/or the above claim 11 - The method described in any one of 17.
An electronic device, comprising the video encoder as claimed in claim 25, a memory and a communication interface, the video encoder executing the method as claimed in claim 18 or claim 19 above.
A computer-readable storage medium, wherein a program is stored in the computer-readable storage medium, and when the program is run on the computer, the computer executes the method according to any one of claims 1-19 method.