WO2024082151A1

WO2024082151A1 - Encoding and decoding methods, encoder, decoder, and storage medium

Info

Publication number: WO2024082151A1
Application number: PCT/CN2022/126026
Authority: WO
Inventors: 马展; 王锡宁; 魏红莲
Original assignee: Oppo广东移动通信有限公司
Priority date: 2022-10-18
Filing date: 2022-10-18
Publication date: 2024-04-25

Abstract

Embodiments of the present application provide encoding and decoding methods. At a decoding end, a decoder decodes a code stream to determine shift coefficient information of a current frame; according to a first scan sequence, determines at least one coefficient block from the shift coefficient information; on the basis of the at least one coefficient block, determines a plurality of shift coefficients according to a second scan sequence; and according to the plurality of shift coefficients and a simplified mesh of the current frame, determines a reconstructed original mesh of the current frame. At an encoding end, an encoder, according to a plurality of shift vectors of the current frame, determines a corresponding plurality of shift coefficients; according to the second scan sequence, sequentially traverses the plurality of shift coefficients, to determine at least one coefficient block; on the basis of the at least one coefficient block, determines shift coefficient information of the current frame according to the first scan sequence; and writes the shift coefficient information into a code stream.

Description

Coding and decoding method, encoder, decoder and storage medium

Technical Field

The embodiments of the present application relate to the technical field of grid compression coding, and in particular to a coding and decoding method, an encoder, a decoder, and a storage medium.

Background technique

In the standard reference software of Dynamic Mesh Coding provided by the Moving Picture Experts Group (MPEG), the encoding and decoding of the geometric information of the mesh mainly includes the organization and compression of the shift coefficients corresponding to the original mesh.

However, the currently common organization method of the shift coefficients is not optimal, which will increase the bit rate of the subsequent lossless coding of the shift coefficients and reduce the grid compression performance.

Summary of the invention

The embodiments of the present application provide a coding and decoding method, an encoder, a decoder and a storage medium, which can use a better organization strategy of shift coefficients, thereby reducing the bit rate of the encoded shift coefficients, thereby improving the grid compression performance.

The technical solution of the embodiment of the present application can be implemented as follows:

In a first aspect, an embodiment of the present application provides a decoding method, which is applied to a decoder, and the method includes:

Decode the code stream and determine the shift coefficient information of the current frame;

determining at least one coefficient block from the shifted coefficient information in a first scanning order;

determining a plurality of shifted coefficients based on the at least one coefficient block in a second scanning order;

A reconstructed original grid of the current frame is determined according to the multiple shift coefficients and the simplified grid of the current frame.

In a second aspect, an embodiment of the present application provides a decoding method, which is applied to a decoder, wherein the decoder includes a video decoder and a grid decoder, and the method includes:

The grid decoder is used to decode the code stream and determine the simplified grid of the current frame;

The video decoder is used to execute the decoding method as described in the first aspect.

In a third aspect, an embodiment of the present application provides an encoding method, which is applied to an encoder, and the method includes:

Determining corresponding multiple shift coefficients according to multiple shift vectors of the current frame;

The plurality of shifted coefficients are sequentially traversed in a second scanning order to determine at least one coefficient block;

Determining shift coefficient information of the current frame according to a first scanning order based on the at least one coefficient block;

The shift coefficient information is written into a bit stream.

In a fourth aspect, an embodiment of the present application provides an encoding method, which is applied to an encoder, wherein the encoder includes a video encoder, a grid encoder, and a preprocessor, and the method includes:

The preprocessor is used to generate a simplified grid and a shift vector according to the original grid of the current frame;

The grid encoder is used to encode the simplified grid to generate a code stream of the simplified grid;

The video encoder is used to execute the encoding method as described in the second aspect.

In a fifth aspect, an embodiment of the present application provides an encoder, the encoder comprising: a first determining unit, an encoding unit; wherein,

The first determination unit is configured to determine a plurality of corresponding shift coefficients according to a plurality of shift vectors of the current frame; sequentially traverse the plurality of shift coefficients in accordance with a second scanning order to determine at least one coefficient block; and determine the shift coefficient information of the current frame in accordance with the first scanning order based on the at least one coefficient block;

The encoding unit is configured to write the shift coefficient information into a bit stream.

In a sixth aspect, an embodiment of the present application provides an encoder, the encoder comprising: a first memory and a first processor; wherein,

The first memory is used to store a computer program that can be run on the first processor;

The first processor is used to execute the method described in the third aspect and the fourth aspect when running the computer program.

In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising: a decoding unit, a second determining unit; wherein,

The decoding unit is configured to decode the code stream;

The second determination unit is configured to determine the shift coefficient information of the current frame; determine at least one coefficient block from the shift coefficient information according to a first scanning order; based on the at least one coefficient block, determine multiple shift coefficients according to a second scanning order; and determine the reconstructed original grid of the current frame based on the multiple shift coefficients and the simplified grid of the current frame.

In an eighth aspect, an embodiment of the present application provides a decoder, a second memory, and a second processor; wherein:

The second memory is used to store a computer program that can be run on the second processor;

The second processor is used to execute the method described in the first aspect and the second aspect when running the computer program.

In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, implements the method described in the first aspect or the second aspect, or implements the method described in the third aspect or the fourth aspect.

The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium. At the decoding end, the decoder decodes the code stream to determine the shift coefficient information of the current frame; determines at least one coefficient block from the shift coefficient information according to the first scanning order; based on the at least one coefficient block, determines multiple shift coefficients according to the second scanning order; determines the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame. At the encoding end, the encoder determines the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame; traverses the multiple shift coefficients in turn according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determines the shift coefficient information of the current frame according to the first scanning order; and writes the shift coefficient information into the code stream. It can be seen that in the embodiment of the present application, when compressing the geometric information of the grid, the codec can traverse the shift coefficients based on the first scanning order to determine the coefficient block, and can traverse the coefficient block based on the second scanning order to determine the shift coefficient information, wherein, in the shift coefficient information of the current frame obtained based on the first scanning order and the second scanning order, the high-frequency information is located at the upper left of the frame, and the low-frequency information is located at the lower right of the frame, so that the high-frequency information with low complexity can be processed first and referenced when the low-frequency information with high complexity is subsequently processed. In other words, in the embodiment of the present application, a better organization strategy of the shift coefficients can be used to reduce the bit rate of the encoded shift coefficients, thereby improving the grid compression performance.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a diagram of the overall framework of grid coding;

Fig. 2 is a schematic diagram of grid preprocessing;

FIG3 is a schematic diagram of intra-frame coding;

FIG4 is a schematic diagram of inter-frame coding;

FIG5 is a schematic diagram of intra-frame decoding;

FIG6 is a schematic diagram of inter-frame decoding;

FIG. 7 is a schematic diagram of a network architecture of a codec provided in an embodiment of the present application.

FIG8 is a schematic diagram of a decoding method proposed in an embodiment of the present application;

FIG9 is a schematic diagram of shift coefficient information;

FIG10 is a schematic diagram 1 of video filling;

FIG11 is a second schematic diagram of video filling;

FIG12 is a schematic diagram of frame filling 1;

FIG13 is a second schematic diagram of frame padding;

FIG14 is a schematic diagram of a raster scanning sequence;

FIG15 is a schematic diagram of a first scanning sequence;

FIG16 is a schematic diagram of a unit block;

Figure 17 is a schematic diagram of a coefficient block

FIG18 is a second schematic diagram of a coefficient block;

FIG19 is a schematic diagram of a zigzag scanning sequence;

FIG20 is a schematic diagram of a second scanning sequence;

FIG21 is a schematic diagram of shift coefficients;

Figure 22 is a schematic diagram of tissue displacement coefficient 1

FIG. 23 is a second schematic diagram of tissue displacement coefficient;

FIG. 24 is a third schematic diagram of tissue displacement coefficient;

FIG25 is a schematic diagram of an encoding method proposed in an embodiment of the present application;

FIG26 is a schematic diagram of the structure of the encoder;

FIG27 is a second schematic diagram of the structure of the encoder;

FIG28 is a schematic diagram of the structure of a decoder;

FIG. 29 is a second schematic diagram of the composition structure of the decoder.

Detailed ways

In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

It should be noted that different data format bitstreams can be decoded and synthesized in the same video scene. Among them, at least image format, point cloud format, and mesh format can be included. In this way, real-time immersive video interaction services can be provided for multiple data formats (for example, mesh, point cloud, image, etc.) with different sources.

In the embodiment of the present application, the data format-based method can allow independent processing at the bitstream level of the data format. That is, like tiles or slices in video encoding, different data formats in the scene can be encoded in an independent manner, so that independent encoding and decoding can be performed based on the data format.

Generally speaking, 3D animation content is represented based on keyframes, that is, each frame is a static mesh. Static meshes at different times have the same topological structure and different geometric structures. However, the amount of data of 3D dynamic meshes represented based on keyframes is extremely large, so how to effectively store, transmit and draw them has become a problem faced by the development of 3D dynamic meshes. In addition, the spatial scalability of the mesh needs to be supported for different user terminals (computers, notebooks, portable devices, mobile phones); different network bandwidths (broadband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, 3D dynamic mesh compression is a very critical issue.

Current 3D dynamic mesh compression methods include space-time prediction methods, which improve compression efficiency by eliminating spatial and temporal correlations; principal component analysis (PCA)-based technology, which projects in the eigenvector space to concentrate energy; and wavelet-based methods, which support spatial scalability and quality scalability.

It should be noted that FIG1 is a diagram of the overall framework of mesh coding, and FIG2 is a schematic diagram of mesh preprocessing. As shown in FIG1 and FIG2, the encoding end is mainly divided into two parts: preprocessing (Pre-processing) and encoder (Encoder). Among them, in the preprocessing process, the original mesh (Original Mesh) is first simplified to obtain a simplified mesh (Decimated Mesh), or a base mesh (Base Mesh). Then the simplified mesh is subdivided to obtain a subdivided mesh (Subdivided Mesh). Finally, for each vertex in the subdivided mesh, find the point in the original mesh that is closest to it, and calculate the displacement vector (Displacement) of the two points. After preprocessing, the simplified mesh and the displacement vector are input into the encoder to generate a bitstream.

Furthermore, during the encoding process, the encoder can be divided into an intra-frame encoder and an inter-frame encoder according to the type of the frame it acts on, which are used to perform intra-frame encoding and inter-frame encoding respectively.

FIG3 is a schematic diagram of intra-frame coding. As shown in FIG3, in the intra-frame encoder, a common static mesh encoder (Static Mesh Encoder) can be used to encode the simplified mesh to generate the corresponding bitstream (Compressed base mesh bitstream). Next, the displacement vector (Update Displacements) is updated with the reconstructed simplified mesh. The updated displacement vector is subjected to wavelet transform (Wavelet Transform) to obtain the displacement coefficient. It is then packaged into images and videos (Image Packing, Video Packing) and encoded using High Efficiency Video Coding (H.265-HEVC) to generate a bitstream (Compressed displacements bitstream) of the displacement coefficient. For attribute map encoding, the feature map is first transformed (Texture Transfer) according to the difference between the reconstructed geometric information and the original geometric information, and then padded (Padding), packaged (Video Packing), and encoded using a video encoder to form an attribute bitstream (Compressed attribute bitstream).

Figure 4 is a schematic diagram of inter-frame coding. As shown in Figure 4, the inter-frame encoder and intra-frame encoder processes are roughly the same, but the inter-frame encoder does not directly encode the simplified grid, but encodes the motion vector between the simplified grid of the current frame and the simplified grid of the reference frame (Motion Encoder), and generates the corresponding motion vector bitstream (Compressed motion bitstream).

Correspondingly, during the decoding process, the decoder can also be divided into an intra-frame decoder and an inter-frame decoder according to the type of the frame it acts on, which are used to perform intra-frame decoding and inter-frame decoding respectively.

FIG5 is a schematic diagram of intra-frame decoding. As shown in FIG5, in the intra-frame decoder, a static mesh decoder (Static Mesh Decoder) can be used to decode the simplified mesh. The video decoder (Video Decoder) is used to decode the shift coefficient video, and the shift coefficient is obtained through video unpacking (Video Unpacking) and inverse wavelet transform (Inverse Wavelet Transform). The decoded mesh geometry information is obtained by decoding the simplified mesh and shift coefficient. The decoding of the attribute graph is directly decoded by the video decoder.

FIG6 is a schematic diagram of inter-frame decoding. As shown in FIG6, for an inter-frame decoder, the process is basically the same as that of an intra-frame decoder, except that the simplified grid is not decoded directly, but the motion vector is decoded, and the simplified grid of the current frame is calculated by the simplified grid of the previous frame (reference frame).

In summary, in the standard reference software for dynamic mesh coding (Dynamic Mesh Coding) provided by the Moving Picture Experts Group (MPEG) (hereinafter referred to as the standard reference software), the geometric information encoding process is divided into the following steps:

1. Simplify the original mesh, specifically by reducing the number of vertices in the mesh and simplifying the connection relationship.

2. Subdivide the simplified mesh in step 1. For any two connected vertices in step 1, add a new point at the midpoint of the line segment connecting them, and repeat twice.

3. For each vertex in step 2, find the point in the original mesh that is closest to it and calculate the displacement vector between the two points.

4. Use an existing static network encoder such as Draco to encode the simplified grid in step 1.

5. Adjust the shift vector in step 3 according to the reconstructed simplified grid obtained in step 4.

6. Perform wavelet transform on the shift vector in step 5, and organize the shift vector after wavelet transform (hereinafter referred to as shift coefficient) into a video.

7. Use a standard video encoder such as H.265 to losslessly encode the shift coefficients in step 6.

It should be noted that when the reference software performs wavelet transform on the shift vector and organizes the shift coefficients obtained by the transform into a video, the shift coefficients are organized in order from low-frequency coefficients to high-frequency coefficients. Specifically, the reference software traverses the transformed shift vector in order from low frequency to high frequency, and organizes it into 16×16 square blocks in the order of Z-scan, and finally organizes the square blocks into video frames in the order of raster scan.

However, in experiments, the above-mentioned organization of shift coefficients is not optimal, which will increase the bit rate of subsequent lossless coding of shift coefficients and reduce the grid compression performance.

In order to solve the above problems, an embodiment of the present application provides a coding and decoding method. At the decoding end, the decoder decodes the bitstream to determine the shift coefficient information of the current frame; determines at least one coefficient block from the shift coefficient information according to the first scanning order; based on the at least one coefficient block, determines multiple shift coefficients according to the second scanning order; determines the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame. At the encoding end, the encoder determines the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame; traverses the multiple shift coefficients in turn according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determines the shift coefficient information of the current frame according to the first scanning order; and writes the shift coefficient information into the bitstream. It can be seen that in the embodiment of the present application, when compressing the geometric information of the grid, the codec can traverse the shift coefficients based on the first scanning order to determine the coefficient block, and can traverse the coefficient block based on the second scanning order to determine the shift coefficient information, wherein, in the shift coefficient information of the current frame obtained based on the first scanning order and the second scanning order, the high-frequency information is located at the upper left of the frame, and the low-frequency information is located at the lower right of the frame, so that the high-frequency information with low complexity can be processed first and referenced when the low-frequency information with high complexity is subsequently processed. In other words, in the embodiment of the present application, a better organization strategy of the shift coefficients can be used to reduce the bit rate of the encoded shift coefficients, thereby improving the grid compression performance.

The embodiment of the present application provides a network architecture of a codec system including a decoding method and an encoding method, and FIG. 7 is a schematic diagram of a network architecture of a codec provided by the embodiment of the present application. As shown in FIG. 7, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with codec functions, for example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., and the embodiment of the present application is not limited. Among them, the decoder or encoder in the embodiment of the present application can be the above-mentioned electronic device.

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

The embodiment of the present application proposes a decoding method. FIG8 is a schematic diagram of the decoding method proposed in the embodiment of the present application. As shown in FIG8, in the embodiment of the present application, the method for the decoder to perform decoding processing may include the following steps:

Step 101: Decode the code stream to determine the shift coefficient information of the current frame.

In an embodiment of the present application, the decoder may first decode the code stream, so as to determine the shift coefficient information of the current frame.

It should be noted that, in the embodiments of the present application, the decoder may be a video decoder, or may be any decoding device including a video decoder and a grid decoder.

Furthermore, in an embodiment of the present application, the code stream transmitted to the decoder may be a code stream of shift coefficients, or may be code stream data including a code stream of shift coefficients and a code stream of a simplified grid (or a code stream of motion vectors).

It is understandable that, in the embodiment of the present application, the current frame may be a current image frame or a current video frame, which is not specifically limited in the present application.

It should be noted that, in an embodiment of the present application, the shift coefficient information of the current frame may include at least one level of details (LOD) composed of at least one coefficient block.

It is understandable that, in the embodiment of the present application, at least one level detail in the shift coefficient information may be arranged in order from low frequency to high frequency, wherein the level detail including high frequency information is located to the left and/or above the level detail including low frequency information.

Exemplarily, in an embodiment of the present application, FIG9 is a schematic diagram of the shift coefficient information. As shown in FIG9, in the shift coefficient information composed of three levels of detail (LOD1, LOD2, LOD3), LOD1 includes the lowest frequency information, LOD2 includes higher frequency information, and LOD3 includes the highest frequency information. Among them, LOD1 including the lowest frequency information is the most complex, while LOD3 including the highest frequency information is the simplest, and LOD3 including the highest frequency information is located on the upper left side of LOD1 including the lowest frequency information.

Furthermore, in an embodiment of the present application, for the shift coefficient information determined by the decoded code stream, if the number of rows of the shift coefficient information is less than a preset height threshold, then it is possible to choose to perform video padding (Video Padding) on the shift coefficient information according to a preset value and a preset height threshold.

It can be understood that in an embodiment of the present application, the preset value can be an integer greater than 0, for example, the preset value is 512, and the preset height threshold can be the maximum height in the grid sequence, that is, the height (number of rows) corresponding to a frame with the largest number of displacement coefficients in the grid sequence.

Exemplarily, in an embodiment of the present application, if the number of rows of the shift coefficient information of the current frame is less than the number of rows corresponding to the frame with the largest number of displacement coefficients in the grid sequence, that is, less than the preset height threshold, then you can choose to use the constant 512 for video padding, so that the height (number of rows) of each frame in the grid sequence can be the same, that is, the height of each frame in the grid sequence is guaranteed to be constant.

It should be noted that in the embodiments of the present application, in the standard reference software, the width of the displacement coefficient video frame (or image frame) can be a fixed constant, and for a grid sequence, the number of displacement coefficients in each frame is not necessarily equal, so that the heights of the video frames (or image frames) formed by different frames are not necessarily equal. Therefore, it is necessary to fill the video frames (or image frames) with smaller heights to ensure that the height of each frame in the displacement coefficient video is constant.

It is understandable that in the embodiment of the present application, the first position may be a position above the last LOD in the shift coefficient information. This is because in the present application, the level details in the shift coefficient information are arranged in the order from low frequency to high frequency in the lower right to upper left positions, so when performing video filling, it is possible to choose to perform filling processing above the last level detail including high frequency information.

Exemplarily, in an embodiment of the present application, Figure 10 is a schematic diagram of video filling. As shown in Figure 10, common shift coefficient information is arranged in order from low frequency to high frequency, and the hierarchical details are arranged from the upper left to the lower right. Then, when performing video filling, the shift coefficient information of the current frame can be filled at the bottom.

Exemplarily, in an embodiment of the present application, Figure 11 is a second schematic diagram of video filling. As shown in Figure 11, the shift coefficient information in the present application is arranged in order from low frequency to high frequency, and the hierarchical details are arranged in sequence from the lower right to the upper left. Then, when performing video filling, the shift coefficient information of the current frame can be filled at the top.

Furthermore, in an embodiment of the present application, for the shift coefficient information determined by the decoded code stream, if there is a blank portion in the last row of the shift coefficient information, the shift coefficient information can be frame padded (Frame Padding) based on the second position according to a preset value.

It can be understood that, in the embodiment of the present application, the preset value may be an integer greater than 0, for example, the preset value is 512.

Exemplarily, in an embodiment of the present application, if there is a blank portion in the last row of the shift coefficient information of the current frame, then the constant 512 can be selected for frame padding, so that the shift coefficient information corresponding to each frame in the grid sequence is rectangular, that is, the rectangular shift coefficient information is ensured to be composed of at least one LOD.

It can be understood that in the embodiment of the present application, the second position can be the upper left position of the last LOD in the shift coefficient information. This is because in the present application, the level details in the shift coefficient information are arranged in the order from low frequency to high frequency in the lower right to upper left positions, so when performing frame filling, the upper left position of the last level detail including high frequency information can be selected for filling processing.

Exemplarily, in an embodiment of the present application, Figure 12 is a schematic diagram of frame filling. As shown in Figure 12, common shift coefficient information arranges the hierarchical details from the upper left to the lower right in order from low frequency to high frequency. Then, when performing frame filling, the missing part in the lower right corner of the shift coefficient information of the current frame can be filled.

Exemplarily, in an embodiment of the present application, Figure 13 is a second schematic diagram of frame filling. As shown in Figure 13, the shift coefficient information in the present application is arranged in order from low frequency to high frequency, and the hierarchical details are arranged in sequence from the lower right to the upper left. Then, when performing frame filling, the missing part in the upper left corner of the shift coefficient information of the current frame can be filled.

Step 102: Determine at least one coefficient block from the shifted coefficient information according to a first scanning order.

In an embodiment of the present application, after determining the shift coefficient information of the current frame by decoding the code stream, the decoder may further determine at least one coefficient block from the shift coefficient information according to the first scanning order.

It should be noted that, in the embodiment of the present application, based on the arrangement order of the level details in the shift coefficient information, the decoder can sequentially traverse the level details including the low-frequency information and the level details including the high-frequency information in the first scanning order.

It is understandable that in the embodiment of the present application, the first scanning order may be the reverse scanning order of the raster scanning order. Raster scanning means scanning from left to right and from top to bottom, first scanning one row and then moving to the starting position of the next row to continue scanning. The common encoding and decoding process mainly uses the raster scanning order.

It should be noted that, in the embodiments of the present application, when the hierarchical details are arranged from the upper left to the lower right in order from low frequency to high frequency, the raster scanning order can often be used to scan from the upper left to the lower right; and in the present application, the hierarchical details in the shift coefficient information are arranged from the lower right to the upper left in order from low frequency to high frequency. Accordingly, when performing a traversal scan, you can choose to scan from the lower right to the upper left in the opposite scanning order of the raster scanning order.

Exemplarily, in an embodiment of the present application, Figure 14 is a schematic diagram of the raster scanning order. As shown in Figure 14, if the level details are arranged from the upper left to the lower right in order from low frequency to high frequency, then LOD1, LOD2, and LOD3 can be traversed in sequence from the upper left to the lower right in accordance with the raster scanning order.

Exemplarily, in an embodiment of the present application, Figure 15 is a schematic diagram of the first scanning order. As shown in Figure 15, if the hierarchical details are arranged from the lower right to the upper left in order from low frequency to high frequency, then LOD1, LOD2, and LOD3 can be traversed in sequence from the lower right to the upper left according to the first scanning order.

It can be understood that in the embodiment of the present application, since each level of detail is composed of at least one coefficient block, at least one coefficient block can be determined after traversing all the level details in the shift coefficient information in the first scanning order.

It can be understood that, in the embodiments of the present application, the coefficient block may be a square block including at least one unit block; wherein each unit block may be composed of 2×2 shift coefficients.

Exemplarily, in an embodiment of the present application, Figure 16 is a schematic diagram of a unit block. As shown in Figure 16, each unit block may include 4 shift coefficients, such as shift coefficients 1-4, wherein the 4 shift coefficients are arranged in 2 rows and 2 columns to form a square unit block.

Further, in an embodiment of the present application, the shift coefficients in the unit block may be arranged in order from low frequency to high frequency, wherein the high frequency shift coefficients are located to the left and/or above the low frequency shift coefficients.

Exemplarily, in an embodiment of the present application, as shown in FIG16 , among the four shift coefficients in the unit block, shift coefficients 1-4 are arranged from low frequency to high frequency in sequence, so shift coefficient 1 is located in the lower right corner and shift coefficient 4 is located in the upper left corner.

It should be noted that, in the embodiments of the present application, the size of the coefficient block can be any integer multiple of the unit block, and the present application does not make any specific limitation.

Exemplarily, in an embodiment of the present application, Figure 17 is a schematic diagram 1 of a coefficient block, and Figure 18 is a schematic diagram 2 of a coefficient block. As shown in Figures 17 and 18, the size of the coefficient block can be 16×16, that is, composed of 4 unit blocks, or 32×32, that is, composed of 16 unit blocks.

Step 103: Determine a plurality of shift coefficients based on at least one coefficient block in a second scanning order.

In an embodiment of the present application, after determining at least one coefficient block from the shift coefficient information according to a first scanning order, the decoder may further determine a plurality of shift coefficients according to a second scanning order based on the at least one coefficient block.

Further, in an embodiment of the present application, based on the arrangement order of the shift coefficients in the coefficient block, the decoder may sequentially traverse the low-frequency shift coefficients and the high-frequency shift coefficients in the coefficient block according to the second scanning order.

It should be noted that, in the embodiment of the present application, the second scanning order may be the reverse scanning order of the zigzag scanning order. The Z in the zigzag scanning (Z-Scan) is a figurative representation, and the zigzag scanning order ensures that different partitions can be addressed in the same traversal order, which is conducive to the recursive implementation in the program.

It should be noted that, in the embodiments of the present application, when the unit blocks composed of the shift coefficients are arranged in order from the upper left to the lower right in order from low frequency to high frequency, a Z-shaped scanning order is often used to scan from the upper left to the lower right; while in the present application, the shift coefficients of the unit blocks in the coefficient block are arranged in order from the lower right to the upper left in order from low frequency to high frequency. Accordingly, when performing a traversal scan, you can choose to scan from the lower right to the upper left in the opposite scanning order of the Z-shaped scanning order.

Exemplarily, in an embodiment of the present application, FIG. 19 is a schematic diagram of a zigzag scanning sequence. As shown in FIG. 19 , if the unit blocks composed of the shift coefficients are arranged in order from the upper left to the lower right in order from low frequency to high frequency, then each unit block and the shift coefficients in each unit block can be traversed in order from the upper left to the lower right in accordance with the zigzag scanning sequence. That is, the scanning of the unit blocks in the coefficient block is performed in a zigzag scanning sequence, and the scanning of the shift coefficients in each unit block is also performed in a zigzag scanning sequence.

Exemplarily, in an embodiment of the present application, FIG. 20 is a schematic diagram of the second scanning order. As shown in FIG. 20, if the unit blocks composed of the shift coefficients are arranged in order from the lower right to the upper left in order from low frequency to high frequency, then each unit block and the shift coefficients in each unit block can be traversed in order from the lower right to the upper left in accordance with the second scanning order. That is, the scanning of the unit blocks in the coefficient block is performed in accordance with the second scanning order, and the scanning of the shift coefficients in each unit block is also performed in accordance with the second scanning order.

It can be understood that in the embodiments of the present application, since the unit blocks in each coefficient block are composed of at least one shift coefficient (such as 4 shift coefficients), multiple shift coefficients can be determined after traversing all the unit blocks in the coefficient block according to the second scanning order.

Step 104: Determine a reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame.

In an embodiment of the present application, after determining multiple shift coefficients according to the second scanning order based on at least one coefficient block, the decoder can further determine the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame.

Furthermore, in an embodiment of the present application, after determining the multiple shift coefficients corresponding to the current frame, the corresponding multiple shift vectors can be further determined based on the multiple shift coefficients; then, the geometric information can be reconstructed based on the multiple shift vectors and the simplified grid to determine the reconstructed original grid of the current frame.

It should be noted that, in the embodiment of the present application, since the shift coefficient is generated by the shift vector after wavelet transformation, it is possible to select to perform wavelet inverse transformation on multiple shift coefficients respectively, so as to determine multiple corresponding shift vectors.

It should be noted that in an embodiment of the present application, when reconstructing geometric information based on multiple shift vectors and a simplified grid of a current frame, the simplified grid can be subdivided first to determine the subdivided grid of the current frame; then the reconstructed original grid can be determined according to the multiple shift vectors and the subdivided grids.

It can be understood that in an embodiment of the present application, the shift vector is obtained through the original grid and the subdivided grid of the current frame. Therefore, after determining the shift vector and the subdivided grid, the geometric information can be further reconstructed based on the shift vector and the subdivided grid to obtain the reconstructed original grid corresponding to the current frame.

Further, in an embodiment of the present application, the simplified grid of the current frame may be obtained through a grid decoder, wherein the grid decoder decodes the bitstream, thereby directly or indirectly determining the simplified grid corresponding to the current frame.

It is understandable that, in the embodiments of the present application, the coding and decoding method proposed in the present application can be applied to both intra-frame coding and decoding and inter-frame coding and decoding, which is not specifically limited in the present application.

It should be noted that, in the embodiment of the present application, for intra-frame coding and decoding, the grid decoder can decode the code stream to obtain a simplified grid of the corresponding current frame.

Accordingly, in an embodiment of the present application, for intra-frame coding and decoding, the grid decoder can receive a code stream of a simplified grid transmitted by the encoding end, and determine a simplified grid of the current frame by decoding the code stream of the simplified grid.

It should be noted that, in an embodiment of the present application, for inter-frame coding and decoding, the grid decoder can decode the code stream to obtain the corresponding motion vector of the current frame, and then can further determine the simplified grid of the current frame based on the motion vector.

Correspondingly, in an embodiment of the present application, for inter-frame coding and decoding, the grid decoder can receive the code stream of the motion vector transmitted by the encoding end, and determine the motion vector of the current frame by decoding the code stream of the motion vector, and then use the motion vector of the current frame and the simplified grid of the decoded previous frame (reference frame) to further determine the simplified grid of the current frame.

In summary, the decoding method proposed by the above steps 101 to 104 adopts a method of organizing shift coefficients in dynamic grid encoding and decoding, changes the organization order of shift coefficients after wavelet transform, and uses a reverse Z-shaped scanning order and an order opposite to the raster scanning order to organize the shift coefficients, thereby reducing the bit rate required for lossless coding of shift coefficients.

It should be noted that, in order to demonstrate the beneficial effects of the coding and decoding method proposed in the embodiment of the present application in practical applications, the shift coefficient organization method proposed in the embodiment of the present application was tested in the MPEG test sequence and compared with the method adopted by the standard reference software. The comparison results are shown in Table 1. The test results of this scheme in the MPEG test sequence are shown in Table 1. In the table, AI and RA respectively represent the All Intra coding mode and the Random Access coding mode. R1-R5 are the five bit rate points specified in the MPEG general test conditions. It can be seen from Table 1 that compared with the shift coefficient organization method adopted by the standard reference software, the shift coefficient organization method proposed in the present embodiment can reduce the bit rate required for encoding the displacement coefficient in R1-R3, and slightly increase R4-R5. At the same time, the scheme described in the present invention does not increase the complexity of coding and decoding, so it has practical value.

Table 1

The embodiment of the present application proposes a decoding method, at the decoding end, the decoder decodes the code stream to determine the shift coefficient information of the current frame; determines at least one coefficient block from the shift coefficient information according to the first scanning order; based on the at least one coefficient block, determines multiple shift coefficients according to the second scanning order; and determines the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame. It can be seen that in the embodiment of the present application, when compressing the geometric information of the grid, the codec can traverse the shift coefficient based on the first scanning order to determine the coefficient block, and can traverse the coefficient block based on the second scanning order to determine the shift coefficient information, wherein, in the shift coefficient information of the current frame obtained based on the first scanning order and the second scanning order, the high-frequency information is located at the upper left of the frame, and the low-frequency information is located at the lower right of the frame, so that the high-frequency information with low complexity can be processed first, and referenced when the low-frequency information with high complexity is subsequently processed. That is to say, in the embodiment of the present application, a better organization strategy of the shift coefficient can be used to reduce the code rate of the encoded shift coefficient, thereby improving the grid compression performance.

Based on the above embodiments, yet another embodiment of the present application proposes a decoding method, which is applied to a decoder, wherein the decoder may include a video decoder and a grid decoder.

It should be noted that, in the embodiments of the present application, the decoding method can be used for intra-frame decoding or inter-frame decoding, which is not specifically limited in the present application.

Further, in an embodiment of the present application, the grid decoder can be used to decode the code stream and determine the simplified grid of the current frame. Wherein, when performing intra-frame decoding, the code stream transmitted to the grid decoder can be a code stream of a simplified grid. In this case, the grid decoder can decode the code stream to obtain the corresponding simplified grid of the current frame. When performing inter-frame decoding, the code stream transmitted to the grid decoder can be a code stream of motion vectors. In this case, the grid decoder can decode the code stream to determine the motion vector of the current frame, and then the motion vector of the current frame and the simplified grid of the decoded previous frame (reference frame) can be used to further determine the simplified grid of the current frame.

It should be noted that, in the embodiment of the present application, after the simplified grid of the current frame is determined by the grid decoder, the video decoder can use the simplified grid to further complete the reconstruction of the original grid of the current frame. The video decoder can first decode the bitstream to determine the shift coefficient information of the current frame; determine at least one coefficient block from the shift coefficient information according to a first scanning order; determine multiple shift coefficients according to a second scanning order based on the at least one coefficient block; and determine the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame.

It can be understood that, in the embodiment of the present application, the first scanning order includes the reverse scanning order of the raster scanning order, and the second scanning order includes the reverse scanning order of the zigzag scanning order.

Further, in an embodiment of the present application, the decoder may determine a plurality of corresponding shift vectors according to the plurality of shift coefficients, and then determine the reconstructed original grid based on the plurality of shift vectors and the simplified grid. The decoder may perform inverse wavelet transform processing on the plurality of shift coefficients to determine the plurality of shift vectors.

It can be understood that, in an embodiment of the present application, the decoder may first subdivide the simplified grid to determine the subdivided grid of the current frame; and then determine the reconstructed original grid according to the multiple shift vectors and the subdivided grid.

It should be noted that, in the embodiment of the present application, the coefficient block is a square block including at least one unit block; wherein the unit block is composed of 2×2 shift coefficients. The shift coefficients in the unit block are arranged in order from low frequency to high frequency, wherein the high frequency shift coefficient is located to the left and/or above the low frequency shift coefficient.

It should be noted that, in an embodiment of the present application, the shift coefficient information includes at least one LOD composed of the at least one coefficient block. The at least one LOD is arranged in order from low frequency to high frequency, wherein the LOD including high frequency information is located on the left and/or above the LOD including low frequency information.

Further, in an embodiment of the present application, if the number of rows of the shift coefficient information is less than a preset height threshold, the shift coefficient information is video padded according to a preset value and the preset height threshold; wherein the first position includes a position above the last LOD in the shift coefficient information.

Furthermore, in an embodiment of the present application, if there is a blank portion in the last row of the shift coefficient information, the shift coefficient information is frame-filled according to the preset value based on the second position; wherein the second position includes the upper left position of the last LOD in the shift coefficient information.

It can be seen that the coding and decoding method proposed in the embodiment of the present application can be a method for organizing shift coefficients in dynamic grid coding and decoding, which reduces the bit rate required for lossless coding shift vectors (shift coefficients) by changing the organization order of shift vectors (shift coefficients) after wavelet transform in the standard reference software.

It can be understood that in the embodiments of the present application, when organizing the shift coefficients, the shift vectors (shift coefficients) after the wavelet transform can be traversed in order from low frequency to high frequency, and organized into 16×16 square blocks (coefficient blocks) in the order of reverse Z-shaped scanning (second scanning order). Then, the organized square blocks can be spliced in the opposite order of raster scanning (first scanning order, i.e., from the lower right to the upper left). Finally, the empty part of the top row is filled with a constant 512 (preset value) to form a rectangular video frame. At the same time, if the number of rows of the current video frame is less than the maximum number of rows of all video frames in the sequence (preset height threshold), the empty part is filled with a constant 512 (preset value).

Exemplarily, in an embodiment of the present application, FIG. 21 is a schematic diagram of a shift coefficient, and FIG. 22 is a schematic diagram of organizing a shift coefficient. As shown in FIG. 21 and FIG. 22, a reverse Z-shaped scanning method may be used in a coefficient block. FIG. 21 traverses the shift coefficients in the order from low-frequency coefficients to high-frequency coefficients, and FIG. 22 shows that the shift coefficients are organized in the order of reverse Z-shaped scanning in a 16×16 (unit block is 4×4) block.

Exemplarily, in an embodiment of the present application, FIG. 23 is a schematic diagram of the tissue shift coefficient 2, and FIG. 24 is a schematic diagram of the tissue shift coefficient 3. As shown in FIG. 23 and FIG. 24, LOD (Level of Details) 1-3 are shift coefficients containing different frequency information, respectively. LOD1 contains the lowest frequency information, and LOD3 contains the highest frequency information. It can be seen that the part containing the low-frequency information is more complex, while the part containing the high-frequency information is simpler.

The method currently used by the common standard reference software is shown in Figure 23, which uses a raster scanning order, that is, starting from the upper left corner, and organizing the shift coefficients from low frequency to high frequency in the order from upper left to lower right. The embodiment of the present application uses the opposite order of raster scanning, starting from the lower right corner, and organizing the shift coefficients from low frequency to high frequency in the order from lower right to upper left.

Since the square blocks included in LOD1-3 may not necessarily form a rectangular video frame, the empty parts need to be filled. Accordingly, when performing frame padding, in the standard reference software, as shown in FIG. 23 , the empty part in the lower right corner is filled, while in the embodiment of the present application, the empty part in the upper left corner is filled.

For a grid sequence, the number of displacement coefficients of each frame is not necessarily equal, which results in that for each frame, the height of the video frame formed after the displacement coefficient organization is not necessarily equal (in the standard reference software, the width of the displacement coefficient video frame is a fixed constant). Therefore, it is necessary to pad the video frames with smaller heights to ensure that the height of each frame in the displacement coefficient video is constant. Accordingly, when performing video padding, in the standard reference software, as shown in FIG. 23 , the video frame is padded at the bottom, while in the embodiment of the present application, the video frame is padded at the bottom.

The embodiment of the present application proposes an encoding method. FIG. 25 is a schematic diagram of the encoding method proposed in the embodiment of the present application. As shown in FIG. 25 , in the embodiment of the present application, the method for the encoder to perform encoding processing may include the following steps:

Step 201: Determine a plurality of corresponding shift coefficients according to a plurality of shift vectors of a current frame.

In an embodiment of the present application, the encoder may first determine a plurality of corresponding shift coefficients of the current frame according to a plurality of shift vectors of the current frame.

It should be noted that, in the embodiments of the present application, the encoder may be a video encoder, or may be any encoding device including a video encoder and a grid encoder.

It is understandable that, in the embodiments of the present application, the current frame may be a current image frame or a current video frame, which is not specifically limited in the present application.

Further, in an embodiment of the present application, when using a shift vector to determine a corresponding shift coefficient, the encoder may choose to perform wavelet transform processing on a plurality of shift vectors respectively, so as to determine a plurality of shift coefficients.

It should be noted that, in an embodiment of the present application, the shift vector can be determined by the refined grid of the current frame and the original grid of the current frame; wherein the original grid is simplified to determine the corresponding simplified grid, and the simplified grid is refined to determine the corresponding refined grid.

It is understandable that in the embodiment of the present application, in the preprocessing process, the original mesh of the current frame can be simplified to obtain a simplified mesh (Decimated Mesh), or a base mesh (Base Mesh). Then the simplified mesh can be subdivided to obtain a subdivided mesh (Subdivided Mesh). Finally, for each vertex in the subdivided mesh, the point closest to it in the original mesh is found, and the displacement vector (Displacement) of the two points is calculated.

Furthermore, in the embodiments of the present application, the coding and decoding method proposed in the present application can be applied to both intra-frame coding and decoding and inter-frame coding and decoding. The present application does not make any specific limitation.

It should be noted that, in the embodiments of the present application, for intra-frame coding and decoding, the grid encoder can encode the simplified grid of the current frame to obtain a code stream of the simplified grid.

Correspondingly, in an embodiment of the present application, for intra-frame coding and decoding, the video encoder may first determine the reconstructed simplified network of the current frame using the bitstream of the simplified grid that has been generated; and then the shift vector may be updated by reconstructing the simplified grid.

That is to say, in an embodiment of the present application, after the grid encoder completes encoding of the simplified grid of the current frame and generates a code stream of the simplified grid, the video encoder can use the code stream of the simplified grid to complete the reconstruction of the simplified grid, and use the reconstructed simplified grid to complete the update of the shift vector.

It should be noted that, in the embodiment of the present application, for inter-frame coding and decoding, the grid encoder can encode the motion vector of the current frame to obtain a code stream of the motion vector.

Correspondingly, in an embodiment of the present application, for inter-frame coding and decoding, the video encoder can first use the bit stream of the motion vector that has been generated to determine the motion vector of the current frame, and then further determine the reconstructed simplified network of the current frame based on the motion vector; finally, the shift vector can be updated by reconstructing the simplified grid.

That is to say, in an embodiment of the present application, after the grid encoder completes encoding of the motion vector of the current frame and generates a bit stream of the motion vector, the video encoder can use the bit stream of the motion vector to reconstruct the simplified grid, and use the reconstructed simplified grid to update the shift vector.

It can be understood that in an embodiment of the present application, the encoder can use the motion vector of the current frame and the reconstructed simplified grid of the reconstructed previous frame (reference frame) to further determine the reconstructed simplified grid of the current frame.

Step 202: traverse a plurality of shift coefficients in sequence according to a second scanning order to determine at least one coefficient block.

In an embodiment of the present application, after determining the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame, the encoder may further sequentially traverse the multiple shift coefficients according to the second scanning order to determine at least one coefficient block.

Exemplarily, in an embodiment of the present application, as shown in FIG. 16 , each unit block may include 4 shift coefficients, such as shift coefficients 1-4, wherein the 4 shift coefficients are arranged in 2 rows and 2 columns to form a square unit block.

Exemplarily, in an embodiment of the present application, as shown in FIG. 17 and FIG. 18 , the size of the coefficient block may be 16×16, ie, composed of 4 unit blocks, or 32×32, ie, composed of 16 unit blocks.

Further, in an embodiment of the present application, the encoder may sequentially traverse multiple shift coefficients of the current frame in a second scanning order from low-frequency shift coefficients to high-frequency shift coefficients, thereby obtaining at least one coefficient block of the current frame.

It should be noted that, in an embodiment of the present application, when the shift coefficients are organized from upper left to lower right in a Z-shaped scanning order in the order from low frequency to high frequency, the unit blocks composed of the shift coefficients can be arranged in sequence from upper left to lower right; whereas in the present application, the shift coefficients of the unit blocks in the coefficient block are arranged in sequence from lower right to upper left in the order from low frequency to high frequency. Accordingly, when traversing the shift coefficients, you can choose to organize the shift coefficients from lower right to upper left in the opposite scanning order of the Z-shaped scanning order.

Exemplarily, in an embodiment of the present application, as shown in FIG19 , multiple shift coefficients are traversed in a zigzag scanning order based on the order from low frequency to high frequency, and the unit blocks composed of the shift coefficients are arranged in turn from the upper left to the lower right. That is, the organization of the unit blocks in the coefficient block is carried out in a zigzag scanning order, and the organization of the shift coefficients in each unit block is also carried out in a zigzag scanning order.

Exemplarily, in an embodiment of the present application, as shown in FIG20 , based on the order from low frequency to high frequency, a plurality of shift coefficients are traversed according to the second scanning order, and the unit blocks composed of the shift coefficients are arranged in the lower right to upper left positions in sequence. That is, the organization of the unit blocks in the coefficient block is carried out according to the second scanning order, and the organization of the shift coefficients in each unit block is also carried out according to the second scanning order.

It can be understood that in the embodiments of the present application, since the unit block in each coefficient block is composed of at least one shift coefficient (such as 4 shift coefficients), after traversing multiple shift coefficients of the current frame according to the second scanning order, a unit block can be formed, and then a coefficient block can be formed.

Step 203: Determine shift coefficient information of the current frame based on at least one coefficient block in a first scanning order.

In an embodiment of the present application, after traversing multiple shift coefficients in sequence according to the second scanning order to determine at least one coefficient block, the encoder can further determine the shift coefficient information of the current frame according to the first scanning order based on the at least one coefficient block.

It should be noted that, in an embodiment of the present application, the shift coefficient information of the current frame may include at least one level of detail LOD composed of at least one coefficient block.

Exemplarily, in an embodiment of the present application, as shown in Fig. 9, in the shift coefficient information composed of three levels of detail (LOD1, LOD2, LOD3), LOD1 includes the lowest frequency information, LOD2 includes higher frequency information, and LOD3 includes the highest frequency information. Among them, LOD1 including the lowest frequency information is the most complex, while LOD3 including the highest frequency information is the simplest, and LOD3 including the highest frequency information is located on the upper left side of LOD1 including the lowest frequency information.

It should be noted that in an embodiment of the present application, based on the order from low frequency to high frequency, the encoder can traverse at least one coefficient block in sequence according to the first scanning order, and finally generate shift coefficient information including hierarchical details of low-frequency information and hierarchical details of high-frequency information.

It should be noted that, in an embodiment of the present application, when at least one coefficient block is traversed from the upper left to the lower right in order from low frequency to high frequency using a raster scanning order, the hierarchical details can be arranged in order from the upper left to the lower right; whereas in the present application, the hierarchical details in the shifted coefficient information are arranged in order from the lower right to the upper left in order from low frequency to high frequency. Accordingly, when organizing the traversal of at least one coefficient block, it is possible to choose to traverse at least one coefficient block from the lower right to the upper left in the reverse scanning order of the raster scanning order.

Exemplarily, in an embodiment of the present application, as shown in FIG14 , if LOD1, LOD2, and LOD3 are organized in sequence from upper left to lower right in the order of raster scanning from low frequency to high frequency, the hierarchical details can be arranged in sequence from upper left to lower right.

Exemplarily, in an embodiment of the present application, as shown in FIG15 , if LOD1, LOD2, and LOD3 are organized in sequence from the lower right to the upper left in the order from low frequency to high frequency according to the first scanning order, the hierarchical details can be arranged in sequence from the lower right to the upper left.

It can be understood that in the embodiment of the present application, since each level of detail is composed of at least one coefficient block, after traversing at least one coefficient block in the first scanning order, the shift coefficient information including at least one level of detail can be determined.

Furthermore, in an embodiment of the present application, for the shift coefficient information of the current frame generated by the encoder, if there is a blank portion in the last line of the shift coefficient information, the shift coefficient information can be frame padded (Frame Padding) based on the second position according to a preset value.

Exemplarily, in an embodiment of the present application, as shown in FIG12 , common shift coefficient information is arranged in order from low frequency to high frequency, with hierarchical details arranged from the upper left to the lower right. When performing frame filling, the vacant part in the lower right corner of the shift coefficient information of the current frame can be filled.

Exemplarily, in an embodiment of the present application, as shown in Figure 13, the shift coefficient information in the present application is arranged in order from low frequency to high frequency, and the hierarchical details are arranged in sequence from the lower right to the upper left. Then, when filling the frame, the missing part in the upper left corner of the shift coefficient information of the current frame can be filled.

Furthermore, in an embodiment of the present application, for the shift coefficient information of the current frame generated by the encoder, if the number of rows of the shift coefficient information is less than a preset height threshold, then it is possible to choose to perform video padding (Video Padding) on the shift coefficient information according to a preset value and a preset height threshold.

Exemplarily, in an embodiment of the present application, as shown in FIG10 , common shift coefficient information is arranged in order from low frequency to high frequency, with hierarchical details arranged from the upper left to the lower right. Then, when performing video filling, the shift coefficient information of the current frame can be filled at the bottom.

Exemplarily, in an embodiment of the present application, as shown in FIG11 , the shift coefficient information in the present application is arranged in order from low frequency to high frequency, and the hierarchical details are arranged from the lower right to the upper left. Then, when performing video filling, the shift coefficient information of the current frame can be filled at the top.

Step 204: Write the shift coefficient information into the bit stream.

In an embodiment of the present application, after determining the shift coefficient information of the current frame according to the first scanning order based on at least one coefficient block, the encoder can further write the shift coefficient information into the bitstream, generate a corresponding bitstream, and transmit it to the decoding end.

It should be noted that, in the embodiment of the present application, the code stream transmitted from the encoder to the decoder may be a code stream of shift coefficients, or may be code stream data including a code stream of shift coefficients and a code stream of a simplified grid (or a code stream of motion vectors).

In summary, the decoding method proposed by the above steps 201 to 204 adopts a method of organizing shift coefficients in dynamic grid encoding and decoding, changes the organization order of shift coefficients after wavelet transform, and uses a reverse Z-shaped scanning order and an order opposite to the raster scanning order to organize the shift coefficients, thereby reducing the bit rate required for lossless coding of shift coefficients.

The embodiment of the present application proposes a coding method, at the coding end, the encoder determines the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame; traverses the multiple shift coefficients in sequence according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determines the shift coefficient information of the current frame according to the first scanning order; writes the shift coefficient information into the code stream. It can be seen that in the embodiment of the present application, when the codec compresses the geometric information of the grid, it can traverse the shift coefficients based on the first scanning order to determine the coefficient block, and can traverse the coefficient block based on the second scanning order to determine the shift coefficient information, wherein, in the shift coefficient information of the current frame obtained based on the first scanning order and the second scanning order, the high-frequency information is located at the upper left of the frame, and the low-frequency information is located at the lower right of the frame, so that the high-frequency information with low complexity can be processed first, and referenced when the low-frequency information with high complexity is subsequently processed. That is to say, in the embodiment of the present application, a better organization strategy of the shift coefficient can be used to reduce the code rate of the encoded shift coefficient, thereby improving the grid compression performance.

Based on the above embodiments, another embodiment of the present application proposes an encoding method, which is applied to an encoder, wherein the encoder includes a video encoder, a grid encoder, and a preprocessor.

It should be noted that, in the embodiments of the present application, the encoding method can be used for intra-frame encoding or inter-frame encoding, which is not specifically limited in the present application.

It should be noted that, in the embodiments of the present application, the preprocessor may be used to generate a simplified grid and a shift vector according to the original grid of the current frame.

It is understandable that in the embodiment of the present application, during the preprocessing process, the original mesh of the current frame can be simplified to obtain a simplified mesh (Decimated Mesh), or a base mesh (Base Mesh). Then the simplified mesh can be subdivided to obtain a subdivided mesh (Subdivided Mesh). Finally, for each vertex in the subdivided mesh, the point closest to it in the original mesh is found, and the displacement vector (Displacement) of the two points is calculated.

Further, in an embodiment of the present application, after the preprocessor generates a corresponding simplified grid based on the original grid, the grid encoder can be used to encode the simplified grid and then generate a code stream of the simplified grid.

It should be noted that, in the embodiments of the present application, for intra-frame coding, the grid encoder can encode the simplified grid of the current frame to obtain a code stream of the simplified grid.

Correspondingly, in an embodiment of the present application, for intra-frame coding, the video encoder may first determine the reconstructed simplified network of the current frame using the bitstream of the simplified grid that has been generated; and then the shift vector may be updated by reconstructing the simplified grid.

It should be noted that, in the embodiment of the present application, for inter-frame coding, the grid encoder can encode the motion vector of the current frame to obtain a code stream of the motion vector.

Correspondingly, in an embodiment of the present application, for inter-frame coding, the video encoder can first use the code stream of the motion vector that has been generated to determine the motion vector of the current frame, and then further determine the reconstructed simplified network of the current frame based on the motion vector; finally, the shift vector can be updated by reconstructing the simplified grid.

It can be understood that, in the embodiment of the present application, after the grid encoder generates the code stream of the simplified grid or the code stream of the motion vector, it can transmit the code stream of the simplified grid or the code stream of the motion vector to the decoding end.

Furthermore, in an embodiment of the present application, the video encoder can be used to determine multiple shift coefficients corresponding to multiple shift vectors of the current frame; traverse the multiple shift coefficients in sequence according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determine the shift coefficient information of the current frame according to the first scanning order; and write the shift coefficient information into the bitstream.

It should be noted that, in the embodiment of the present application, the encoder may perform wavelet transform processing on the multiple shift vectors, so as to determine the multiple shift coefficients.

It can be understood that in an embodiment of the present application, the shift vector is determined by the refined grid of the current frame and the original grid of the current frame; wherein the original grid is simplified to determine the corresponding simplified grid, and the simplified grid is refined to determine the corresponding refined grid.

Exemplarily, in an embodiment of the present application, as shown in Figures 21 and 22, a reverse Z-scanning method may be used within a coefficient block. Figure 21 shows traversing the shift coefficients in the order from low-frequency coefficients to high-frequency coefficients, and Figure 22 shows organizing the shift coefficients in the order of reverse Z-scanning within a 16×16 (unit block is 4×4) block.

Exemplarily, in the embodiments of the present application, as shown in FIG. 23 and FIG. 24 , LOD (Level of Details) 1-3 are shift coefficients containing different frequency information, respectively. LOD1 contains the lowest frequency information, and LOD3 contains the highest frequency information. It can be seen that the part containing the low frequency information is more complex, while the part containing the high frequency information is simpler.

Based on the above embodiment, in another embodiment of the present application, based on the same inventive concept as the above embodiment, FIG. 26 is a schematic diagram of a composition structure of an encoder. As shown in FIG. 26, the encoder 110 may include: a first determining unit 111, an encoding unit 112, wherein:

The first determining unit 111 is configured to determine a plurality of corresponding shift coefficients according to a plurality of shift vectors of the current frame; sequentially traverse the plurality of shift coefficients in accordance with the second scanning order to determine at least one coefficient block; and determine the shift coefficient information of the current frame in accordance with the first scanning order based on the at least one coefficient block;

The encoding unit 112 is configured to write the shift coefficient information into a bit stream.

It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 110. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, the method described in any one of the aforementioned embodiments is implemented.

Based on the composition of the above-mentioned encoder 110 and the computer-readable storage medium, Figure 27 is a second schematic diagram of the composition structure of the encoder. As shown in Figure 27, the encoder 110 may include: a first memory 113 and a first processor 114, a first communication interface 115 and a first bus system 116. The first memory 113, the first processor 114, and the first communication interface 115 are coupled together through the first bus system 116. It can be understood that the first bus system 116 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 116 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 116 in Figure 10. Among them,

The first communication interface 115 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

The first memory 113 is used to store a computer program that can be run on the first processor;

The first processor 114 is used to determine the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame when running the computer program; traverse the multiple shift coefficients in sequence according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determine the shift coefficient information of the current frame according to the first scanning order; and write the shift coefficient information into the bit stream.

It can be understood that the first memory 113 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). The first memory 113 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

The first processor 114 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 114. The above-mentioned first processor 114 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 113, and the first processor 114 reads the information in the first memory 113 and completes the steps of the above method in combination with its hardware.

It is understood that the embodiments described in this application can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application or a combination thereof. For software implementation, the technology described in this application can be implemented by a module (such as a process, function, etc.) that performs the functions described in this application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

Optionally, as another embodiment, the first processor 114 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

FIG. 28 is a schematic diagram of a first structure of a decoder. As shown in FIG. 28 , the decoder 120 may include: a decoding unit 121 and a second determining unit 122; wherein,

The decoding unit 121 is configured to decode the code stream;

The second determination unit 122 is configured to determine the shift coefficient information of the current frame; determine at least one coefficient block from the shift coefficient information according to a first scanning order; based on the at least one coefficient block, determine multiple shift coefficients according to a second scanning order; and determine the reconstructed original grid of the current frame based on the multiple shift coefficients and the simplified grid of the current frame.

Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 120. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, the method described in any one of the above embodiments is implemented.

Based on the composition of the above-mentioned decoder 120 and the computer-readable storage medium, Figure 29 is a second schematic diagram of the composition structure of the decoder. As shown in Figure 29, the decoder 120 may include: a second memory 123 and a second processor 124, a second communication interface 125 and a second bus system 126. The second memory 123 and the second processor 124, and the second communication interface 125 are coupled together through the second bus system 126. It can be understood that the second bus system 126 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 126 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 126 in Figure 12. Among them,

The second communication interface 125 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

The second memory 123 is used to store a computer program that can be run on the second processor;

The second processor 124 is used to decode the code stream and determine the shift coefficient information of the current frame when running the computer program; determine at least one coefficient block from the shift coefficient information according to a first scanning order; based on the at least one coefficient block, determine multiple shift coefficients according to a second scanning order; determine the reconstructed original grid of the current frame based on the multiple shift coefficients and the simplified grid of the current frame.

It can be understood that the second memory 123 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). The second memory 123 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

The second processor 124 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the second processor 124. The above-mentioned second processor 124 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the second memory 123, and the second processor 124 reads the information in the second memory 123 and completes the steps of the above method in combination with its hardware.

It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Industrial Applicability

The embodiment of the present application provides a coding and decoding method, an encoder, a decoder and a storage medium. At the decoding end, the decoder decodes the code stream to determine the shift coefficient information of the current frame; determines at least one coefficient block from the shift coefficient information according to the first scanning order; based on the at least one coefficient block, determines multiple shift coefficients according to the second scanning order; determines the reconstructed original grid of the current frame according to the multiple shift coefficients and the simplified grid of the current frame. At the encoding end, the encoder determines the corresponding multiple shift coefficients according to the multiple shift vectors of the current frame; traverses the multiple shift coefficients in turn according to the second scanning order to determine at least one coefficient block; based on the at least one coefficient block, determines the shift coefficient information of the current frame according to the first scanning order; and writes the shift coefficient information into the code stream. It can be seen that in the embodiment of the present application, when compressing the geometric information of the grid, the codec can traverse the shift coefficients based on the first scanning order to determine the coefficient block, and can traverse the coefficient block based on the second scanning order to determine the shift coefficient information, wherein the high-frequency information is located at the upper left of the frame and the low-frequency information is located at the lower right of the frame in the shift coefficient information of the current frame obtained based on the first scanning order and the second scanning order, so that the high-frequency information with low complexity can be processed first and referred to in the subsequent processing of the low-frequency information with high complexity. In other words, in the embodiment of the present application, a better organization strategy of the shift coefficient can be used to reduce the bit rate of the encoded shift coefficient, thereby improving the grid compression performance.

Claims

A decoding method, applied to a decoder, comprising:

Decode the code stream and determine the shift coefficient information of the current frame;

determining at least one coefficient block from the shifted coefficient information in a first scanning order;

determining a plurality of shifted coefficients based on the at least one coefficient block in a second scanning order;

A reconstructed original grid of the current frame is determined according to the multiple shift coefficients and the simplified grid of the current frame.
The method according to claim 1, wherein

The first scanning order comprises a reverse scanning order of a raster scanning order.
The method according to claim 1, wherein

The second scanning order includes a reverse scanning order of the zigzag scanning order.
The method according to claim 1, wherein the method further comprises:

Determine a corresponding plurality of shift vectors according to the plurality of shift coefficients;

The reconstructed original mesh is determined based on the plurality of shift vectors and the simplified mesh.
The method according to claim 4, wherein the method further comprises:

Perform inverse wavelet transform processing on the multiple shift coefficients to determine the multiple shift vectors.
The method according to claim 4, wherein the method further comprises:

Subdividing the simplified grid to determine a subdivided grid of the current frame;

The reconstructed original grid is determined according to the plurality of shift vectors and the subdivided grid.
The method according to claim 1, wherein

The coefficient block is a square block including at least one unit block; wherein the unit block is composed of 2×2 shift coefficients.
The method according to claim 7, wherein:

The shift coefficients in the unit block are arranged in order from low frequency to high frequency, wherein the high frequency shift coefficients are located on the left and/or above the low frequency shift coefficients.
The method according to claim 7 or 8, wherein

The shift coefficient information includes at least one level of detail (LOD) composed of the at least one coefficient block.
The method according to claim 9, wherein

The at least one LOD is arranged in order from low frequency to high frequency, wherein the LOD including high frequency information is located on the left and/or upper side of the LOD including low frequency information.
The method according to claim 10, wherein the method further comprises:

If the number of rows of the shift coefficient information is less than a preset height threshold, the shift coefficient information is video padded according to a preset value and the preset height threshold; wherein the first position includes a position above the last LOD in the shift coefficient information.
The method according to claim 10, wherein the method further comprises:

If there is a blank portion in the last row of the shift coefficient information, the shift coefficient information is frame-filled according to the preset value based on the second position; wherein the second position includes the upper left position of the last LOD in the shift coefficient information.
The method according to claim 1, wherein the method further comprises:

The simplified grid is obtained by decoding the code stream through a grid decoder.
The method according to claim 1, wherein the method further comprises:

The code stream is decoded by a grid decoder to obtain the motion vector, and the simplified grid is determined based on the motion vector.
A decoding method is applied to a decoder, wherein the decoder includes a video decoder and a trellis decoder, and the method includes:

The grid decoder is used to decode the code stream and determine the simplified grid of the current frame;

The video decoder is used to execute the decoding method as described in any one of claims 1-14.
A coding method, applied to an encoder, comprising:

Determining corresponding multiple shift coefficients according to multiple shift vectors of the current frame;

The plurality of shifted coefficients are sequentially traversed in a second scanning order to determine at least one coefficient block;

Determining shift coefficient information of the current frame according to a first scanning order based on the at least one coefficient block;

The shift coefficient information is written into a bit stream.
The method according to claim 16, wherein

The first scanning order comprises a reverse scanning order of a raster scanning order.
The method according to claim 16, wherein

The second scanning order includes a reverse scanning order of the zigzag scanning order.
The method according to claim 16, wherein the method further comprises:

The plurality of shift vectors are subjected to wavelet transform processing to determine the plurality of shift coefficients.
The method according to claim 16, wherein

The coefficient block is a square block including at least one unit block; wherein the unit block is composed of 2×2 shift coefficients.
The method according to claim 20, wherein

The shift coefficients in the unit block are arranged in order from low frequency to high frequency, wherein the high frequency shift coefficients are located on the left and/or above the low frequency shift coefficients.
The method according to claim 20 or 21, wherein the method further comprises:

The shift coefficient information includes at least one LOD constituted by the at least one coefficient block.
The method according to claim 22, wherein the method further comprises:

The at least one LOD is arranged in order from low frequency to high frequency, wherein the LOD including high frequency information is located on the left and/or upper side of the LOD including low frequency information.
The method according to claim 23, wherein the method further comprises:

If the number of rows of the shift coefficient information is less than a preset height threshold, the shift coefficient information is video padded according to a preset value and the preset height threshold; wherein the first position includes a position above the last LOD in the shift coefficient information.
The method according to claim 23, wherein the method further comprises:

If there is a blank portion in the last row of the shift coefficient information, the shift coefficient information is frame-filled according to the preset value based on the second position; wherein the second position includes the upper left position of the last LOD in the shift coefficient information.
The method according to claim 16, wherein

The shift vector is determined by the refined grid of the current frame and the original grid of the current frame; wherein the original grid is simplified to determine the corresponding simplified grid, and the simplified grid is refined to determine the corresponding refined grid.
The method according to claim 26, wherein the method further comprises:

Determine a reconstructed simplified network of the current frame using a code stream of a simplified grid of the current frame;

The displacement vector is updated by reconstructing the simplified grid.
The method according to claim 26, wherein the method further comprises:

Determine the motion vector of the current frame using the code stream of the motion vector of the current frame, and determine the reconstructed simplified network of the current frame according to the motion vector;

The displacement vector is updated by reconstructing the simplified grid.
A coding method, applied to an encoder, wherein the encoder includes a video encoder, a trellis encoder and a preprocessor, the method comprising:

The preprocessor is used to generate a simplified grid and a shift vector according to the original grid of the current frame;

The grid encoder is used to encode the simplified grid to generate a code stream of the simplified grid;

The video encoder is used to execute the encoding method as described in any one of claims 16-27.
An encoder, comprising: a first determining unit, an encoding unit; wherein:

The first determination unit is configured to determine a plurality of corresponding shift coefficients according to a plurality of shift vectors of the current frame; sequentially traverse the plurality of shift coefficients in accordance with a second scanning order to determine at least one coefficient block; and determine the shift coefficient information of the current frame in accordance with the first scanning order based on the at least one coefficient block;

The encoding unit is configured to write the shift coefficient information into a bit stream.
An encoder comprises: a first memory and a first processor; wherein:

The first memory is used to store a computer program that can be run on the first processor;

The first processor is configured to execute the method according to any one of claims 16 to 28 or 29 when running the computer program.
A decoder, comprising: a decoding unit, a second determining unit; wherein:

The decoding unit is configured to decode the code stream;

The second determination unit is configured to determine the shift coefficient information of the current frame; determine at least one coefficient block from the shift coefficient information according to a first scanning order; based on the at least one coefficient block, determine multiple shift coefficients according to a second scanning order; and determine the reconstructed original grid of the current frame based on the multiple shift coefficients and the simplified grid of the current frame.
A decoder, comprising: a second memory and a second processor; wherein:

The second memory is used to store a computer program that can be run on the second processor;

The second processor is configured to execute the method according to any one of claims 1 to 14 or 15 when running the computer program.
A computer-readable storage medium storing a computer program, wherein the computer program, when executed, implements the method according to any one of claims 1 to 14, or 15, or implements the method according to any one of claims 16 to 28, or 29.