WO2023193709A1

WO2023193709A1 - Coding method and apparatus, decoding method and apparatus, and device

Info

Publication number: WO2023193709A1
Application number: PCT/CN2023/086202
Authority: WO
Inventors: 邹文杰; 张伟; 杨付正; 吕卓逸
Original assignee: 维沃移动通信有限公司
Priority date: 2022-04-08
Filing date: 2023-04-04
Publication date: 2023-10-12
Also published as: CN116934880A

Abstract

The present application relates to the technical field of coding and decoding. Disclosed are a coding method and apparatus, a decoding method and apparatus, and a device. The coding method comprises: a coding end performing decoding and reconstruction on geometric information of a coded target three-dimensional grid, so as to acquire reconstructed first geometric information; the coding end performing repeated vertex screening on the first geometric information, so as to acquire information of repeated vertexes, wherein the repeated vertexes are vertexes among a plurality of vertexes having the same position coordinates other than a first vertex, and the first vertex is one of the plurality of vertexes having the same position coordinates; the coding end re-sorting a first connection relationship of the target three-dimensional grid according to the information of the repeated vertexes, so as to acquire a re-sorted second connection relationship; and the coding end coding the second connection relationship.

Description

Encoding and decoding methods, devices and equipment

Cross-references to related applications

This application claims priority to Chinese Patent Application No. 202210370125.8 filed in China on April 8, 2022, the entire content of which is incorporated herein by reference.

Technical field

This application belongs to the field of coding and decoding technology, and specifically relates to a coding and decoding method, device and equipment.

Background technique

Considering that the vertices and point clouds of a three-dimensional grid are both a set of discrete points randomly distributed in space, they have similar characteristics. Therefore, the geometric information in the three-dimensional mesh can also be compressed using video-based point cloud compression standards. However, the vertices of the three-dimensional grid have the characteristics of sparser and more uneven spatial distribution, and the compression efficiency of directly using video-based point cloud compression methods is low. In addition, the video-based point cloud compression standard does not involve the compression of mesh vertex connection relationships, mesh texture (Ultraviolet, UV) coordinate attributes and texture maps. Due to the generation mechanism of the three-dimensional mesh, there will be some geometric repeating points. These geometric repeating points have the same geometric coordinates but different UV coordinates. The geometric repeating points will cause the discontinuity problem of the three-dimensional mesh, and there will be single triangles with many sides adjacent to each other. Side cases, and unilateral needs to consume redundant codewords to identify, thus reducing the coding efficiency of connectivity relationships.

Contents of the invention

The embodiments of the present application provide an encoding and decoding method, device and equipment, which can solve the problem that the three-dimensional mesh generation mechanism of the existing technology will cause some geometric repeating points. These geometric repeating points will cause the three-dimensional mesh to be discontinuous, resulting in the reduction of the three-dimensional network. The issue of encoding efficiency of lattice connectivity relationships.

The first aspect provides an encoding method, including:

The encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid, and obtains the reconstructed first geometric information;

The encoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates, and the first vertex is the position coordinate. One of the same vertices;

The encoding end reorders the first connection relationships of the target three-dimensional grid based on the repeated vertex information, and obtains the reordered second connection relationships;

The encoding end encodes the second connection relationship.

In a second aspect, an encoding device is provided, including:

The first acquisition module is used to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain the reconstructed first geometric information;

The second acquisition module is used to filter the repeated vertices of the first geometric information and obtain the repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex Is one of the vertices among multiple vertices with the same position coordinates;

A third acquisition module, configured to reorder the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtain the reordered second connection relationships;

The first encoding module is used to encode the second connection relationship.

The third aspect provides a decoding method, including:

The decoding end decodes and reconstructs the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

The decoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates, and the first vertex is the position coordinate. One of the same vertices;

The decoding end decodes the first connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

In the fourth aspect, a decoding device is provided, including:

The reconstruction module is used to decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

The fifth acquisition module is used to perform repeated vertex screening on the first geometric information and obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex Is one of the vertices among multiple vertices with the same position coordinates;

The first decoding module is used to decode the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

In a fifth aspect, a coding device is provided, including a processor and a memory. The memory stores programs or instructions that can be run on the processor. When the program or instructions are executed by the processor, the first The steps of the method described in this aspect.

In a sixth aspect, an encoding device is provided, including a processor and a communication interface, wherein the processor is used to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain the reconstructed first geometric information; The first geometric information is filtered for repeated vertices to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex is a plurality of vertices with the same position coordinates. one of the vertices in; reorder the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtain the reordered second connection relationship; and encode the second connection relationship.

In a seventh aspect, a decoding device is provided, including a processor and a memory. The memory stores programs or instructions that can be run on the processor. When the program or instructions are executed by the processor, the third process is implemented. The steps of the method described in this aspect.

In an eighth aspect, a decoding device is provided, including a processor and a communication interface, wherein the processor is used to root Decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional mesh; perform repeated vertex screening on the first geometric information to obtain repeated vertex information, and the repeated vertices are multiple vertices with the same position coordinates except Vertices other than the first vertex, the first vertex being one of multiple vertices with the same position coordinates; decoding the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

In a ninth aspect, a communication system is provided, including: an encoding device and a decoding device. The encoding device can be used to perform the steps of the method described in the first aspect, and the decoding device can be used to perform the steps of the method described in the third aspect. steps of the method.

In a tenth aspect, a readable storage medium is provided. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method are implemented as described in the first aspect. The steps of the method described in the third aspect.

In an eleventh aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the method described in the first aspect. method, or implement a method as described in the third aspect.

In a twelfth aspect, a computer program/program product is provided, the computer program/program product is stored in a storage medium, and the computer program/program product is executed by at least one processor to implement as described in the first aspect The steps of the method, or the steps of implementing the method as described in the third aspect.

In the embodiment of the present application, by decoding and reconstructing the geometric information of the encoded target three-dimensional mesh, the reconstructed first geometric information is obtained, and the first geometric information is subjected to repeated vertex screening to obtain repeated vertex information. According to The repeated vertex information is used to reorder the first connection relationship of the target three-dimensional mesh, obtain the reordered second connection relationship, and encode the second connection relationship; the above solution uses the reconstructed geometry The information is screened for repeated vertices, and then the connection relationships are reordered and encoded based on the repeated vertex information. This can avoid the discontinuity problem of the three-dimensional grid caused by repeated vertices and improve the coding efficiency of the connectivity relationships of the three-dimensional grid. .

Description of the drawings

Figure 1 is a schematic flow chart of the encoding method according to the embodiment of the present application;

Figure 2 is a schematic diagram of the process of merging vertices in mesh simplification;

Figure 3 is a schematic diagram of the fine division process based on grid;

Figure 4 is a schematic diagram of the eight directions of patch arrangement;

Figure 5 is a schematic diagram of the encoding process of high-precision geometric information;

Figure 6 is a schematic diagram of raw patch;

Figure 7 is one of the schematic diagrams of coding connection relationships in the embodiment of the present application;

Figure 8 is the second schematic diagram of the coding connection relationship in the embodiment of the present application;

Figure 9 is a geometric diagram of the prediction principle;

Figure 10 is a schematic diagram of the coding framework of a three-dimensional grid based on video;

Figure 11 is a schematic module diagram of an encoding device according to an embodiment of the present application;

Figure 12 is a schematic structural diagram of an encoding device according to an embodiment of the present application;

Figure 13 is a schematic flow chart of the decoding method according to the embodiment of the present application;

Figure 14 is a block diagram of geometric information reconstruction;

Figure 15 is a schematic diagram of the decoding framework based on the three-dimensional grid of video;

Figure 16 is a schematic module diagram of a decoding device according to an embodiment of the present application;

Figure 17 is a schematic structural diagram of a communication device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art fall within the scope of protection of this application.

The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and that "first" and "second" are distinguished objects It is usually one type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the related objects are in an "or" relationship.

It is worth pointing out that the technology described in the embodiments of this application is not limited to Long Term Evolution (LTE)/LTE Evolution (LTE-Advanced, LTE-A) systems, and can also be used in other wireless communication systems, such as code Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access, OFDMA), Single-carrier Frequency Division Multiple Access (SC-FDMA) and other systems. The terms "system" and "network" in the embodiments of this application are often used interchangeably, and the described technology can be used not only for the above-mentioned systems and radio technologies, but also for other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in much of the following description, but these techniques can also be applied to applications other than NR system applications, such as 6th ^generation Generation, 6G) communication system.

The encoding and decoding methods, devices and equipment provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through some embodiments and their application scenarios.

As shown in Figure 1, this embodiment of the present application provides an encoding method, including:

Step 101: The encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid, and obtains the reconstructed first geometric information;

It should be noted that the target three-dimensional grid mentioned in this application can be understood as the three-dimensional grid corresponding to any video frame. The geometric information of the target three-dimensional grid can be understood as the coordinates of the vertices in the three-dimensional grid. These coordinates are usually Refers to three-dimensional coordinates.

Step 102: The encoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information;

It should be noted that by filtering the repeated vertices, the information of the repeated vertices can be obtained, and at the same time, the second geometric information can be obtained. The second geometric information is the mesh geometric information excluding the repeated vertices. The repeated vertices are Vertices other than the first vertex among the plurality of vertices with the same position coordinates, and the first vertex is one of the plurality of vertices with the same position coordinates.

Step 103: The encoding end reorders the first connection relationships of the target three-dimensional grid based on the repeated vertex information, and obtains the reordered second connection relationships;

It should be noted that the connection relationship mentioned in the embodiment of this application is used to describe the connection relationship between elements such as vertices and patches in the three-dimensional mesh, and may also be called a connectivity relationship.

Step 104: The encoding end encodes the second connection relationship.

It should be noted that the embodiment of the present application removes repeated vertices from the reconstructed geometric information, then reorders the connection relationships, and then encodes the reordered connection relationships, thereby avoiding the three-dimensional distortion caused by repeated vertices. The problem of grid discontinuity improves the coding efficiency of the connectivity relationship of the three-dimensional grid.

It should be noted that in order to obtain the reconstructed geometric information, the geometric information needs to be encoded first. In an embodiment of the present application, the specific method of encoding the geometric information of the three-dimensional grid may include the following process:

Step S21: Before encoding geometric information, the target three-dimensional grid can be simplified first to obtain a simplified grid;

It should be noted that this process is an optional step.

Specifically, when performing simplification, the encoding end simplifies the target three-dimensional grid based on quantization parameters to obtain a simplified grid.

It should be noted that the quantization parameters mentioned in this application mainly include quantization parameters in the three components of X direction, Y direction and Z direction.

Optionally, based on the quantified parameters, the target three-dimensional grid is simplified. The specific implementation method of obtaining the simplified grid is:

When merging vertices in the target three-dimensional grid, the encoding end adjusts the position coordinate values of some or all of the merged vertices in the target three-dimensional grid to multiples of quantization parameters to obtain a simplified grid.

It should be noted that by considering the quantization parameter when setting the position of the merged point, and setting it as a multiple of the quantization parameter, the multiple can be any value that meets the requirements, thereby ensuring that no quantization is required during inverse quantization. The additional information can restore the original position, which will reduce the amount of data consumed by high-precision geometric information.

The specific implementation of the simplification process is described below.

For the input original mesh, mesh simplification is performed first. The focus of mesh simplification lies in the simplified operations and the corresponding error measures. The mesh simplification operation here can be edge-based simplification. As shown in Figure 2, the number of patches and vertices can be reduced by merging two vertices of an edge. In addition, the mesh can also be simplified through point-based mesh simplification methods.

During the mesh simplification process it is necessary to define a simplified error measure. For example, you can select the directions of all adjacent faces of a vertex The sum of the process coefficients is used as the error measure of the vertex, and the error measure of the corresponding edge is the sum of the error measures of the two vertices on the edge. After determining the method of simplification operation and the error measure, you can start to simplify the mesh. For example, the mesh can be divided into one or more local meshes, and the vertex error of the initial mesh in the slice is first calculated to obtain the error of each edge. Then all the edges in the piece are arranged according to a certain rule according to the error, such as from small to large. Each simplification can merge edges according to certain rules, such as selecting the edge with the smallest error for merging. At the same time, the merged vertex position is calculated and the errors of all edges related to the merged vertex are updated, and the order of edge arrangement is updated. Iteratively simplify the mesh's faces to some desired number.

The specific process includes:

1. Calculation of vertex error

The vertex error can be defined as the sum of the coefficients of the equations of all adjacent faces of the vertex. For example, each adjacent face defines a plane, which can be expressed by Formula 1:

Formula 1: D ² =(n ^T v+d) ² =v ^T (nn ^T )v+2dn ^T v+d ² ;

Among them, D is the distance from any vertex to the plane, n is the unit normal vector of the plane, v is the position vector of the vertex, and d is a constant. It is expressed as Formula 2 in the form of a quadratic surface: Q=(A,b,c)=(nn ^T ,dn,d ² );

Among them, Q is the vertex error, A, b, c are the coefficients representing the corresponding symbols in Formula 1.

From formula 2, formula 3 is obtained: Q(v)=v ^T Av+2b ^T v+c;

Since the vertex error is the sum of equation coefficients of all adjacent faces of the vertex, formula 4 can be made: Q ₁ (v) + Q ₂ (v) = (Q ₁ + Q ₂ ) (v) = (A ₁ +A ₂ ,b ₁ +b ₂ ,c ₁ +c ₂ )(v). Among them, Q(v) is the vertex error, v is the corresponding vertex, Q ₁ (v) is the equation of v's adjacent plane 1, Q ₂ (v) is the equation of v's adjacent plane 2, A ₁ , A ₂ , b ₁ , b ₂ , c ₁ , c ₂ are their corresponding coefficients. Of course, if there are multiple adjacent surfaces, the corresponding plane error equation can be added to Formula 4.

2. Merge vertices

A major step in the process of merging vertices is determining the location of the merged vertices. According to the error formula three, the vertex position can be selected to make the error as small as possible. For example, by taking the partial derivative of formula 3, we can get formula 4:

Formula five:

It can be seen from the above formula that only when the matrix A is invertible, the point that minimizes the error can be found. Therefore, there are many ways to obtain the merged vertex positions here. If you consider the quality of mesh simplification, if matrix A is reversible, select the vertex position that minimizes the error; if matrix A is irreversible, you can select one of the two endpoints on the edge that minimizes the error. If you consider the complexity of mesh simplification, you can directly select the midpoint of the edge or one of the two endpoints as the position of the merged vertex. If the efficiency of quantization after mesh simplification is considered, the merged vertex positions also need to be adjusted. Since high-precision information needs to be encoded separately after quantization, a part of the merged vertex positions are adjusted to multiples of the corresponding quantization parameters to ensure that the original positions can be restored without additional information during inverse quantization, which will reduce the number of high-precision geometries. The amount of data consumed by the message.

Once you have determined how to select the merged vertex positions, you can begin the process of merging vertices. For example, you can first calculate the errors of all edges in the initial grid and arrange them according to a certain specification, such as from small to large. Every Iterations, select the edge whose error satisfies a certain rule, such as the edge with the smallest error. Removes the two endpoints of an edge from the mesh vertices and adds the merged vertices to the collection of mesh vertices. Use all or part of the adjacent vertices of the two vertices before merging as adjacent vertices of the merged vertex, and then update the error metric of all points connected to the merged vertex to obtain the error of the newly generated edge. Then update the edge order from the global perspective of the slice. The above process is repeated until the number of faces required to meet the lossy encoding is reached.

3. Update the connection relationship

After merging vertices, since some vertices are deleted from the vertex set and many new vertices are added, the connection relationship between the vertices needs to be updated. For example, during the process of merging vertices, it is possible to determine that the merged vertex corresponds to the two vertices before merging. You only need to replace all the indexes of the two vertices before merging that appear in the face with the index of the merged vertex, and then delete the face with duplicate indexes to achieve the purpose of updating the connection relationship.

The above is the main process of mesh simplification. At the same time, the three-dimensional grid may also carry attribute information, and the attribute information may also need to be simplified. For meshes with attribute information, such as texture coordinates, colors, normal vectors, etc., the vertex coordinates can be extended to higher dimensions to calculate the vertex error with attribute information. Taking texture coordinates as an example, assuming the vertex coordinates are (x, y, z) and the texture coordinates are (u, v), then the expanded vertices are (x, y, z, u, v). Assume the expanded triangle T = (p, q, r). In order to determine the error measure in the high-dimensional space, first calculate two standard orthogonal vectors, namely:

Formula six:

Formula 7:

Among them, e ₁ and e ₂ are two vectors on the plane where T is located, q is, e ₂ is, where "·" represents the dot product of the vectors, which defines a coordinate axis on the high-dimensional plane, with p as the origin. Consider an arbitrary point v, and u=pv, according to formula 8: ‖u‖ ² = (u·e ₁ ) ² +...+(μ·e _n ) ² ;

That is Formula 9: (u·e ₃ ) ² +...+(u·e _n ) ² =‖μ‖ ² -(μ·e ₁ ) ² -(u·e ₂ ) ² .

Since e ₁ and e ₂ are two vectors on the plane where T is located, the left-hand term of Formula 9 is the square of the distance from the vertex to the plane where T is located, that is, Formula 10: D ² =‖μ‖ ² -(μ·e ₁ ) ² -(u·e ₂ ) ² ;

After expanding and merging them, an equation similar to Formula 3 can be obtained, where:

Formula 11:

Formula 12: b=(p·e ₁ )e ₁ +(p·e ₂ )e ₂ -p;

Formula 13: c=p·p-(p·e ₁ ) ² -(p·e ₂ ) ² .

After obtaining the above error metric, the same subsequent steps as the previous three-dimensional information can be carried out, thereby achieving the simplification of the grid with attribute information.

Generally speaking, the edge part of an image can attract people's attention more, thus affecting people's evaluation of the quality of the image. The same is true for three-dimensional meshes. People tend to notice the boundaries more easily. Therefore, whether boundaries are maintained is also a factor that affects the quality of mesh simplification. The boundaries of the mesh are generally the boundaries of geometric shapes and textures. When an edge belongs to only one face, the edge is a geometric boundary. When the same vertex has two or more texture coordinates, the vertex is the boundary of the texture coordinates. None of the above boundaries should be merged during mesh simplification. Therefore, during each simplification, you can first determine whether the vertex on the edge is a boundary point. If it is a boundary point, skip it and proceed directly to the next iteration.

Step S22: Quantify the geometric information of the simplified grid to obtain the first information;

It should be noted that the geometric information of the simplified mesh can be understood as the coordinates of the vertices in the three-dimensional mesh, and these coordinates usually refer to three-dimensional coordinates.

Specifically, the first information includes at least one of the following:

A11, first precision geometric information;

It should be noted that the first precision geometric information can be understood as low-precision geometric information, that is, the low-precision geometric information refers to the quantized geometric information of the target three-dimensional grid, that is, the three-dimensional coordinates of each vertex included in the quantized target three-dimensional grid. coordinate information.

A12, second precision geometric information;

It should be noted that the second precision geometric information can be understood as high-precision geometric information, and the high-precision geometric information can be regarded as geometric information lost in the quantization process, that is, lost three-dimensional coordinate information.

A13. Supplementary point information;

It should be noted that the information of supplementary points refers to the information of points that require additional processing generated during the quantification process. That is to say, the supplementary points are points that require additional processing generated during the quantification process. For example, the coordinate positions overlap. By processing the repeated points, the vertices whose coordinate positions overlap during quantization can be restored to their original positions after dequantization.

Optionally, the supplementary point information includes at least one of the following:

A131. Supplement the index of the vertex in the first precision geometric information corresponding to the point;

It should be noted that by identifying the index, you can know which points in the quantized grid identify multiple points in the three-dimensional grid before quantification, that is, multiple points in the three-dimensional grid before quantization are After they are coincident together, the low-precision geometric information of the supplementary points can be determined through the index of the vertices.

A132. Supplement the third precision geometric information of the point;

It should be noted that the third-precision geometric information can be understood as low-precision geometric information of the supplementary points, that is, the quantized three-dimensional coordinate information of the supplementary points.

A133. Supplement the fourth precision geometric information of the point;

It should be noted that the fourth precision geometric information can be understood as the high-precision geometric information of the supplementary point, that is, the three-dimensional coordinate information of the supplementary point that is lost during the quantization process.

What needs to be noted here is that in specific use, the hidden points after quantization can be determined through A131 and A133 or A132 and A133.

Optionally, in an embodiment of the present application, the specific implementation of step S22 is:

The encoding end quantizes each vertex in the simplified grid according to the quantization parameter of each component to obtain first precision geometric information.

It should be noted that the quantization parameters of each component can be flexibly set according to usage requirements.

Normally, for quantization that does not require high precision, only low-precision geometric information can be retained after quantization; while for quantization that requires higher precision, not only low-precision geometric information must be recorded during quantization, but also high-precision geometric information must be recorded. Information, so that accurate grid recovery can be achieved during decoding. That is to say, the specific implementation of the above step S22 should also include:

The encoding end obtains second precision geometric information based on the first precision geometric information and the quantization parameter of each component.

For example, assume that the three-dimensional coordinates of a vertex are (x, y, z), the quantization parameters are (QP _x , QP _y , QP _z ), low-precision geometric information (x _l , y _l , z _l ) and high-precision geometric information The calculation process of (x _h ,y _h ,z _h ) is as shown in Formula 14 to Formula 19:

Formula 14: x _l =f ₁ (x,QP _x );

Formula 15: y _l =f ₁ (y,QP _y );

Formula 16: z _l =f ₁ (z,QP _z );

Formula 17: x _h = f ₂ (x, x _l , QP _x );

Formula 18: y _h = f ₂ (y, y _l ,QP _y );

Formula 19: z _h = f ₂ (z,z _l ,QP _z );

Among them, the f ₁ function in Formula 14 to Formula 16 is a quantization function. The input of the quantization function is the coordinate of a certain dimension and the quantization parameter of that dimension, and the output is the quantized coordinate value; Formula 17 to Formula 19 The input of the f ₂ function in is the original coordinate value, the quantized coordinate value and the quantized parameter of this dimension, and the output is a high-precision coordinate value.

The f ₁ function can be calculated in many ways. A more common calculation method is as shown in Formula 20 to Formula 22. It is calculated by dividing the original coordinates of each dimension by the quantized parameter of that dimension. Among them, / is the division operator, and the result of the division operation can be rounded in different ways, such as rounding, rounding down, rounding up, etc. There are also multiple calculation methods for the f ₂ function. The implementation methods corresponding to Formula 17 to Formula 19 are as shown in Formula 23 to Formula 25, where * is the multiplication operator.

Formula 20: x _l =x/QP _x ;

Formula 21: y _l =y/QP _y ;

Formula 22: z _l =z/QP _z ;

Formula 23: x _h =xx _l *QP _x ;

Formula 24: y _h =yy _l *QP _y ;

Formula 25: z _h =zz _l *QP _z ;

When the quantization parameter is an integer power of 2, the f ₁ function and f ₂ function can be implemented using bit operations, such as Formula 26 to Formula 31:

Formula 26: x _l =x>>log ₂ QP _x ;

Formula 27: y _l =y>>log ₂ QP _y ;

Formula 28: z _l =z>>log ₂ QP _z ;

Formula 29: x _h =x&(QP _x -1);

Formula 30: y _h =y&(QP _y -1);

Formula 31: z _h =z&(QP _z -1);

It is worth noting that no matter which calculation method is used for the f ₁ function and f ₂ function, the quantization parameters QP _x , QP _y and QP _z are all Can be set flexibly. First, the quantization parameters of different components are not necessarily equal. You can use the correlation of the quantization parameters of different components to establish the relationship between QP _x , QP _y and QP _z , and set different quantization parameters for different components; secondly, different spaces The quantization parameters of the regions are not necessarily equal. The quantization parameters can be adaptively set according to the sparsity of the vertex distribution in the local region.

It should be noted that the high-precision geometric information contains detailed information of the outline of the three-dimensional mesh. In order to further improve the compression efficiency, the high-precision geometric information (x _h , y _h , z _h ) can be further processed. In a three-dimensional mesh model, the importance of high-precision geometric information of vertices in different areas is different. For areas where vertices are sparsely distributed, the distortion of high-precision geometric information will not have a major impact on the visual effect of the three-dimensional mesh. At this time, in order to improve the compression efficiency, you can choose to further quantify the high-precision geometric information, or only retain the high-precision geometric information of some points.

Optionally, during the quantization process, there may be multiple quantized points that completely coincide with the same position. That is to say, in this case, the specific implementation of the above step S22 should also include:

The encoding end determines the information of the supplementary point based on the geometric information of the simplified grid and the first precision geometric information.

That is to say, after obtaining the low-precision geometric information of all vertices, the points with repeated low-precision geometric information are used as supplementary points and encoded separately. The geometric information of supplementary points can also be divided into two parts: low-precision geometric information and high-precision geometric information. According to the application's requirements for compression distortion, you can choose to retain all supplementary points or only a part of them. The high-precision geometric information of the supplementary points can also be further quantified, or the high-precision geometric information of only some points can be retained.

It should be noted that after quantizing the geometric information of the target three-dimensional grid to obtain the first information, the first information needs to be encoded to obtain the final code stream. Optionally, the first information is encoded as mentioned in the embodiment of the present application. The specific implementation process of encoding information includes:

Step S221, the encoding end processes the first information to obtain second information, where the second information includes at least one of a placeholder map and a geometric map;

Step S222: The encoding end encodes the second information.

It should be noted that because the types of information contained in the first information are different, when processing the first information, different types of information will be processed separately. The implementation of step S221 will be described below from the perspective of different information. The process is explained below.

1. The first information includes first precision geometric information

Optionally, in this case, the specific implementation process of step S221 includes:

Step S2211, the encoding end divides the first precision geometric information into three-dimensional slices;

It should be noted that in this case, the main step is to divide the low-precision geometric information into patches to obtain multiple three-dimensional patches; the specific implementation method of this step is: the encoding end determines each element contained in the first-precision geometric information. The projection plane of the vertices; the coding end performs slice division on the vertices contained in the first precision geometric information according to the projection plane; the coding end clusters the vertices contained in the first precision geometric information, and obtains Each piece after division. In other words, the process of patch division mainly includes: first estimating the normal vector of each vertex, selecting the plane normal vector and the vertex The candidate projection plane with the smallest angle between the point normal vectors is used as the projection plane of the vertex; then, the vertices are initially divided according to the projection plane, and the vertices with the same and connected projection planes are composed into patches; finally, the fine division algorithm is used to optimize the clustering Similar results are obtained to obtain the final three-dimensional patch (3D patch).

The specific implementation of the process of obtaining the three-dimensional slice from the first-precision geometric information is described in detail below.

First estimate the normal vector of each point. The tangent plane and its corresponding normal are defined based on each point's nearest neighbor vertex m at a predefined search distance. KD tree is used to separate data and find adjacent points near point p _i , the center of gravity of the set Used to define normals. The calculation method of the center of gravity c is as follows:

Formula thirty-two:

Use the eigendecomposition method to estimate the vertex normal vector, and the calculation process is shown in Formula 33:

Formula thirty-three:

In the initial partitioning stage, the projection plane of each vertex is initially selected. Let the estimated value of the vertex normal vector be The normal vector of the candidate projection plane is Select the plane whose normal vector direction is closest to the vertex normal vector direction as the projection plane of the vertex. The calculation process of plane selection is as shown in Equation 34:

Formula thirty-four:

The fine division process can use a grid-based algorithm to reduce the time complexity of the algorithm. The grid-based fine division algorithm flow is shown in Figure 3, which specifically includes:

First set the number of cycles (numlter) to 0 and determine whether the number of cycles is less than the maximum number of cycles (it should be noted that the maximum number of cycles can be set according to usage requirements). If it is less than the maximum number of cycles, perform the following process:

Step S301: Divide the (x, y, z) geometric coordinate space into voxels.

It should be noted that the geometric coordinate space here refers to the geometric coordinate space composed of the first-precision geometric information obtained by quantization. For example, for a 10-bit mesh using a voxel size of 8, the number of voxels at each coordinate would be 1024/8 = 128, and the total number of voxels in this coordinate space would be 128×128×128.

Step S302: Find filled voxels. Filled voxels refer to voxels that contain at least one point in the grid.

Step S303, calculate the smoothing score of each filled voxel on each projection plane, recorded as voxScoreSmooth. The voxel smoothing score of the voxel on a certain projection plane is the number of points gathered to the projection plane through the initial segmentation process.

Step S304, use KD-Tree partitioning to find neighboring filled voxels, recorded as nnFilledVoxels, that is, the nearest filled voxels of each filled voxel (within the search radius and/or limited to the maximum number of adjacent voxels).

Step S305, use the voxel smoothing score of the nearest neighbor filled voxel in each projection plane to calculate the smoothing score (scoreSmooth) of each filled voxel. The calculation process is as shown in Formula 35:

Formula thirty-five:

where p is the index of the projection plane and v is the index of the nearest neighbor filling voxel. The scoreSmooth of all points in a voxel is the same.

Step S306: Calculate the normal score using the normal vector of the vertex and the normal vector of the candidate projection plane, recorded as scoreNormal. The calculation process is as shown in Formula 36:

Formula 36: scoreNormal[i][p]=normal[i]·orientation[p];

where p is the index of the projection plane and i is the index of the vertex.

Step S307, use scoreSmooth and scoreNormal to calculate the final score of each voxel on each projection plane. The calculation process is as shown in Equation 37:

Formula thirty-seven:

Among them, i is the vertex index, p is the index of the projection plane, and v is the voxel index where vertex i is located.

Step S308: Use the scores in step 307 to cluster the vertices to obtain finely divided patches.

Iterate the above process multiple times until a more accurate patch is obtained.

Step S2212: The encoding end performs two-dimensional projection on the divided three-dimensional slice to obtain the two-dimensional slice;

What needs to be said is that this process is to project the 3D patch onto a two-dimensional plane to obtain a two-dimensional patch (2D patch).

Step S2213, the encoding end packages the two-dimensional slices to obtain two-dimensional image information;

It should be noted that this step implements patch packing. The purpose of patch packing is to arrange 2D patches on a two-dimensional image. The basic principle of patch packing is to arrange patches on a two-dimensional image without overlapping or The pixel-free parts of the patch are partially overlapped and arranged on the two-dimensional image. Through priority arrangement, time domain consistent arrangement and other algorithms, the patches are arranged more closely and have time domain consistency to improve coding performance.

Assume that the resolution of the 2D image is WxH, and the minimum block size that defines the patch arrangement is T, which specifies the minimum distance between different patches placed on this 2D grid.

First, patches are inserted and placed on the 2D grid according to the non-overlapping principle. Each patch occupies an area consisting of an integer number of TxT blocks. In addition, there is a requirement of at least one TxT block between adjacent patches. When there is not enough space to place the next patch, the height of the image will be doubled and the patch will continue to be placed.

In order to arrange the patches more closely, the patches can choose a variety of different arrangement directions. For example, eight different arrangement directions can be adopted, as shown in Figure 4, including 0 degrees, 180 degrees, 90 degrees, 270 degrees, and mirror images of the first four directions.

In order to obtain better adaptability to inter-frame prediction characteristics of video encoders, a patch arrangement method with temporal consistency is adopted. In a Group of frame (GOF), all patches of the first frame are arranged in order from largest to smallest. For other frames in the GOF, the temporal consistency algorithm is used to adjust the order of patches.

It should also be noted here that after obtaining the two-dimensional image information, the patch information can be obtained based on the information in the process of obtaining the two-dimensional image information, and then the patch information can be encoded to obtain the patch information sub-stream.

What needs to be explained here is that in the process of obtaining two-dimensional image information, it is necessary to record the information of patch division, the information of patch projection plane and the information of patch packing position, so the patch information records the information of each step operation in the process of obtaining two-dimensional image. , that is, the patch information includes: patch division information, patch projection plane information, and patch packing position information.

Step S2214: The encoding end obtains a first-precision placeholder map and a first-precision geometric map based on the two-dimensional image information;

What needs to be said is that the process of obtaining the placeholder map is mainly: using the patch arrangement information obtained by patch packing, setting the position of the vertex in the two-dimensional image to 1, and setting the remaining positions to 0 to obtain the placeholder map. For the process of obtaining the geometric map, the main process is: in the process of obtaining the 2D patch through projection, the distance from each vertex to the projection plane is saved. This distance is called the depth. The low-precision geometric map compression part is to compress each 2D patch in the 2D patch. The depth value of the vertex is arranged to the position of the vertex in the placeholder map to obtain a low-precision geometric map.

2. The first information includes second precision geometric information

Step S2215, the encoding end obtains the arrangement order of the vertices contained in the first precision geometric information;

Step S2216: The encoding end arranges the second-precision geometric information corresponding to the vertices contained in the first-precision geometric information in the two-dimensional image to generate a second-precision geometric map.

It should be noted that the high-precision geometric information is arranged in the original patch (raw patch), and the high-precision geometric information corresponding to the vertices in the low-precision geometric map is arranged in a two-dimensional image to obtain the raw patch, thereby generating a high-precision Accurate geometric drawings. It is mainly divided into three steps, as shown in Figure 5, including:

Step 501: Obtain the arrangement order of vertices, scan the low-precision geometric map line by line from left to right, and use the scanning order of each vertex as the order of vertices in the raw patch.

Step 502, generate raw patch.

It should be noted that the raw patch is a rectangular patch formed by arranging the three-dimensional coordinates of the vertices row by row as shown in Figure 6. According to the vertex arrangement order obtained in the first step, the high-precision geometric information of the vertices is arranged in order to obtain the high-precision geometric information raw patch.

Step 503: Place the high-precision geometric information in a two-dimensional image to generate a high-precision geometric map.

It should be noted that when encoding to obtain the geometric figure sub-stream, the encoding end will encode the first-precision geometric figure and the second-precision geometric figure to obtain the geometric figure sub-stream.

3. The first information includes information on supplementary points

Step S2217, the encoding end arranges the third precision geometric information of the supplementary points into the first original slice;

Step S2218: The encoding end arranges the fourth precision geometric information of the supplementary points into a second original slice in the same order as the first original slice;

Step S2219: The encoding end compresses the first original slice and the second original slice to obtain a geometric map of supplementary points.

It should be noted that in the embodiment of the present application, the low-precision part and the high-precision part of the geometric information of the supplementary points are encoded separately. First, the low-precision geometric information of the supplementary points is arranged into a supplementary point low-precision raw patch in any order; then, the high-precision geometric information is arranged into a supplementary point high-precision raw patch in the same order as the supplementary point low-precision raw patch; finally , to compress supplementary point low-precision raw patches and high-precision raw patches, a variety of compression methods can be used. Among them, one method is to encode the values in the raw patch using run-length coding, entropy coding, etc. The other method is to add the supplementary point low-precision raw patch to the blank area in the low-precision geometric map, and add the supplementary point low-precision raw patch to the blank area of the low-precision geometric map. Point high-precision raw patches are added to the blank areas in the high-precision geometric map to obtain a geometric map that supplements the points.

It should be noted that after obtaining the encoded geometric information, the geometric information can be reconstructed to obtain the reconstructed first geometric information, and then the first geometric information can be filtered for repeated vertices to obtain repeated vertex information. Optional In one embodiment of the present application, the method of obtaining repeated vertex information includes:

The encoding end obtains the repeated vertices in the first geometric information, and records the index of the repeated vertices to form repeated vertex information;

Wherein, the repeated vertex information includes the index of the repeated vertex and the texture coordinate index of the first vertex and the repeated vertex.

It should be noted that by obtaining repeated vertices one by one and recording their indexes, it is possible to ensure that more accurate information about repeated vertices is obtained.

Optionally, optional implementation methods of step 103 include:

Step 1031: The encoding end rearranges the vertex texture coordinates of the target three-dimensional grid according to the index of the repeated vertex, and obtains the reordered texture coordinate information;

Optionally, the specific implementation of this step is: the encoding end sequentially moves the texture coordinates corresponding to the repeated vertices to the end of the list corresponding to the texture coordinates according to the order in which the repeated vertices appear in the first geometric information. , forming the reordered texture coordinate information.

For example, the texture coordinates of 10 vertices (vertex 1 to vertex 10) from front to back are: coordinate 1, coordinate 2, coordinate 3, coordinate 4, coordinate 5, coordinate 6, coordinate 7, coordinate 8, coordinate 9, coordinate 10 , and vertex 1 and vertex 6 are repeated, then after reordering, the texture coordinates of the 10 vertices from front to back are: coordinate 1, coordinate 2, coordinate 3, coordinate 4, coordinate 5, coordinate 7, coordinate 8, coordinate 9, Coordinate 10, coordinate 6.

It should be noted that by using the order of repeated vertices in the first geometric information to reorder the texture coordinate information, it can be ensured that there are no texture coordinates of repeated vertices in other parts of the reordered texture coordinate information except the tail. Information to facilitate subsequent adjustments to the connection relationship.

Step 1032: The encoding end adjusts the first connection relationship of the target three-dimensional grid according to the reordered texture coordinate information and the repeated vertex information, and obtains the second connection relationship after the reordering;

It should be noted that by reordering the connection relationships, the processed connection relationships do not contain information about repeated vertices. This can avoid the problem of discontinuity in the three-dimensional mesh caused by repeated vertices when encoding the connection relationships, and improve the three-dimensional Encoding efficiency of grid connectivity relations.

It should be noted that since the connection relationship is composed of a texture coordinate index part and a geometry index part, when adjusting the connection relationship, these two parts need to be adjusted separately. Optionally, in an embodiment of the present application, the specific implementation process of step 1032 is:

Step 10321: The encoding end modifies the texture coordinate index part of the first connection relationship of the target three-dimensional grid according to the texture coordinate information after reordering;

Step 10322: The encoding end traverses the geometric index part of the first connection relationship of the target three-dimensional grid, and replaces the index of the repeated vertex of the first connection relationship of the target three-dimensional grid with the second one according to the repeated vertex information. one The index of the vertex, obtains the second connection relationship after reordering.

For example, the connection relationship between 10 vertices (vertex 1 to vertex 10) from front to back is: (1,1), (2,2), (3,3), (4,4), (5,5 ), (6,6), (7,7), (8,8), (9,9), (10,10), where the first value in the brackets represents the geometric index of the vertex, and the last value Texture coordinate index. Since vertex 1 and vertex 6 are repeated, the reordered connection relationship representation from front to back is: (1,1), (2,2), (3,3), (4,4), (5,5), (1,10), (6,6), (7,7), (8,8), (9,9).

It should be noted that the specific implementation process of encoding the second connection relationship in the embodiment of this application is:

Step S31: The encoding end determines the vertices to be sorted within the first spatial range based on the spatial angle between adjacent triangle patches in the three-dimensional grid. The first spatial range is the target vertex of the triangle to be encoded in the three-dimensional grid. The spatial range where the vertex to be sorted includes the target vertex.

The connection relationship encoding in the embodiment of the present application is a connection relationship encoding method driven by geometric information, where the reconstructed geometric information includes index information of vertices in the three-dimensional grid.

In this embodiment of the present application, the above three-dimensional network can be divided into at least one triangular patch, and each triangular patch contains at least one triangle.

In this step, based on the spatial angle between adjacent triangular patches in the three-dimensional grid, some vertices in the first spatial range can be filtered out, and the remaining vertices can be used as vertices to be sorted.

Optionally, the first spatial range includes:

the spatial range between the first sphere and the second sphere;

Wherein, the center of the first sphere and the second sphere are the same, the radius of the first sphere and the radius of the second sphere are different, and the center of the sphere is the target in the first side of the triangle to be encoded. Position, for example, the center of the sphere is the midpoint of the first side of the triangle to be encoded.

Step S32: The encoding end sorts the vertices to be sorted to obtain the sorting information of the target vertices.

In this step, the vertices to be sorted are sorted according to a preset sorting criterion. For example, the sorting criterion can be sorting according to the distance between the vertex and the midpoint of the first side of the triangle to be encoded, or the sorting criterion can also be sorted according to the distance between the vertex and the first side of the triangle to be encoded. Sort by the size of the radius of the circumcircle of the triangle formed by the first side. Of course, the sorting criterion can also be other criteria, which are not specifically limited here.

Here, through the spatial angle between adjacent triangular patches, the vertices in the first spatial range are further deleted, reducing the number of vertices to be sorted, that is, the bit information used in the sorting information of the target vertices can be reduced.

Step S33: The encoding end obtains the encoding information of the triangle to be encoded based on the encoding information corresponding to the sorting information of the target vertex.

Optionally, when the target condition is met, the encoding end encodes the sorting information of the target vertex to obtain the encoding information; when the target condition is not met, the encoding end encodes the index of the target vertex to obtain the above Encoded information. For example, the target condition is that the number of vertices to be sorted in the first spatial range is less than a preset threshold, and/or the sorting number of the target vertex is less than a preset value. Since the number of vertices to be sorted in the first space range is small or the sorting sequence number of the target vertex is small, the encoding information corresponding to the sorting information will occupy a smaller number of bits. At this time Encoding the sorting information of the target vertices can effectively reduce the number of encoding bits. For situations where the number of vertices to be sorted is large or the sorting sequence number of the target vertex is large, encoding the index of the target vertex can effectively reduce the number of encoding bits compared to encoding the sorting information.

In the embodiment of the present application, some vertices are excluded in the first spatial range according to the spatial angle between adjacent triangular patches in the three-dimensional grid, and the vertices to be sorted are determined based on the excluded vertices, that is, the number of vertices to be sorted is reduced. In this way, when encoding the sorting information of the target vertex, the number of bits occupied by the encoding information can be further reduced, thereby effectively improving the encoding efficiency.

Optionally, before the encoding end determines the first spatial range, it also includes:

The encoding end selects the first edge from the edge set corresponding to the three-dimensional grid, wherein the edge set is a set of at least one edge of the encoded triangle in the three-dimensional grid;

The encoding end determines the triangle to be encoded based on the first side and the vertex corresponding to the first side, wherein the target vertex of the triangle to be encoded is the vertex corresponding to the first side divided by the third A vertex other than the two vertices connected by one side, the target vertex can also be described as the opposite vertex of the first side.

Optionally, the encoding end determines the vertices to be sorted within the first spatial range based on the spatial angle between adjacent triangular patches in the three-dimensional grid, including:

When the triangle to be encoded is a triangle outside the preset category of triangles, the encoding end determines the vertices to be sorted within the first spatial range based on the spatial angle between adjacent triangle patches in the three-dimensional grid.

Optionally, the preset category triangle includes at least one of the following:

A triangle whose angle to the encoded triangle is smaller than the preset angle;

A triangle with two vertices coincident or three vertices collinear. Specifically, it means that two vertices in a triangle coincide or three vertices are collinear.

Optionally, the method in the embodiment of this application also includes:

When the triangle to be encoded is the above-mentioned preset category triangle, the encoding end obtains the encoding information of the triangle to be encoded based on the encoding information corresponding to the target vertex information of the triangle to be encoded.

For example, when the triangle to be encoded is the triangle of the preset category, the index of the target vertex of the triangle to be encoded is directly encoded, and the encoding information of the triangle to be encoded is obtained based on the encoding information corresponding to the index of the target vertex.

In the embodiment of the present application, when encoding the index of a vertex, binary representation can be directly used or a coding algorithm such as Huffman can be used for encoding. The encoding method is not specifically limited here.

Optionally, after obtaining the encoding information of the triangle to be encoded, the method further includes:

The encoding end updates the edge set according to the first preset rule;

The encoding end re-determines the triangles to be encoded based on the updated edge set until all triangles in the three-dimensional grid obtain encoding information;

Wherein, the first preset rule includes: adding two sides of the triangle to be encoded except the first side to the side set, and removing the first side from the side set. side.

The encoding end excludes all vertices of the first target triangle from the vertices within the first space range to obtain remaining vertices;

The encoding end determines the vertices to be sorted within the first spatial range based on the remaining vertices;

Wherein, the first target triangle is a triangle whose angle with an adjacent coded triangle is less than the angle threshold, and one side of the first target triangle is the same as the first side of the triangle to be coded.

Optionally, the encoding information of the triangle to be encoded also includes: encoding information of the angle threshold.

Here, by encoding the angle threshold, the decoding end can obtain the angle threshold based on the encoding information, and determine the vertices to be sorted in the first spatial range based on the angle threshold. In this way, the encoding end can flexibly Set the angle threshold.

Of course, a fixed angle threshold can also be pre-agreed. The encoding end and the decoding end determine the vertices to be sorted in the first spatial range based on the pre-agreed angle threshold. The encoding end does not need to encode the angle threshold.

Optionally, the method of the embodiment of the present application further includes: encoding the target vertex information of the triangle to be encoded in a second spatial range to obtain the encoding information of the triangle to be encoded, and the second spatial range is the The range in the three-dimensional grid other than the first spatial range.

Optionally, the encoding information of the triangle to be encoded also includes encoding information of the first spatial range.

For example, by encoding the radii of the above-mentioned first sphere and the second sphere, the first spatial range can be flexibly set in this implementation.

Of course, the encoding end and the decoding end can also predetermine the size of the first spatial range. In this method, the encoding end does not need to encode the first spatial range.

In a specific embodiment of the present application, the input three-dimensional mesh is divided into one or more slices at the encoding end, and an initial triangle is selected in each slice. Encode the vertex index of the initial triangle and put the edges of the initial triangle into the edge set (ie, the edge set). Select an edge in the edge set and determine its pair of vertices. The triangle formed by the edge and the pair of vertices is the triangle to be encoded. For example, the edge selected in each iteration can be recorded as τ, its opposite vertex is recorded as v, and the triangle to be encoded adjacent to the edge can be encoded. As shown in Figures 7 and 8, the process of encoding connection relationships may specifically include:

(1) When the preset conditions are met, the vertex index of the triangle can be directly encoded, or the triangle can be encoded in other ways. The preset condition may be that the triangle to be encoded belongs to several special triangles, such as a degenerate surface (two points coincide or three points are collinear) or the angle between the triangle and the encoded triangle is less than a certain angle, or the preset condition The condition is that the number of vertices in the first space range is greater than the preset number, or the preset condition is that the target vertex is outside the second space range, such as within the second space range, or the target vertex is within the preset condition The sorting numbers of the vertices in the first space range are greater than or equal to the preset value. This preset condition can be flexibly set according to requirements. Add the two sides of the triangle to be encoded except side τ to the edge set, and remove edge τ from the set. Then take out the other side of the triangle to be encoded (the side other than side τ) from the side set according to certain criteria, and continue to encode the triangle adjacent to this side. For example, the next side τ can be selected in the order of access.

(2) If the above preset conditions are not met, determine the spatial range where the vertex v is located (ie, the above-mentioned first spatial range) and encode the spatial range. Traverse all vertices within the space range in the slice, filter out all vertices of the new triangle formed by edge τ and have an angle smaller than a certain angle with the encoded adjacent triangle, and encode the angle value.

The spatial range can be determined using the geometric properties of adjacent triangular patches, spatial angles, or other criteria. For example, the spatial range can be the part between two concentric spheres with the midpoint of side τ as the center of the sphere and the minimum radius R _min and the maximum radius R _max combining {R _min , R _max }. The code is {R _min , R _max } group.

Optionally, the above angle values can also be encoded.

(3) Traverse all vertices within the space range and sort them according to certain sorting criteria. For example, the sorting criterion can be the distance from the vertex v to the midpoint of the side τ; or the radius of the circumscribed circle of the triangle formed by the side τ. Encodes the sequence number of vertex v in the sorting.

(4) Add the two edges of the new coded triangle except the edge τ to the edge set, and remove the edge τ from the edge set. Then take out the other side of the newly encoded triangle from the set according to certain criteria, and continue to encode the triangles adjacent to this side.

This encoding process is iterated for each patch of the three-dimensional mesh until all triangles in each patch are encoded. If the edge set is empty but there are unencoded triangles, an initial triangle is selected from the remaining unencoded triangles and the encoding process is cycled.

It should also be noted that, in order to ensure the consistency of the connection relationship, in one embodiment of the present application, the encoding end has changes in the connection relationship of the target three-dimensional mesh based on the reconstructed first geometric information. Adjustment is made, the second connection relationship after reordering is performed based on the adjusted connection relationship, and then the second connection relationship is encoded.

It should be noted that the embodiments of this application also include encoding of repeated vertices. The specific implementation method is:

In the process of encoding the second connection relationship, the encoding end queries the three vertices of each triangular patch to see whether there are repeated vertices based on the repeated vertex information;

When a duplicate vertex exists, the encoding end identifies the duplicate vertex;

After completing the encoding of the second connection relationship, the encoding end performs encoding according to the identifier of the repeated vertex, and obtains the code stream of the repeated vertex identifier.

It should be noted that by marking the repeated vertices in the connection relationship based on the repeated vertex information, and then using the marks for encoding, the accuracy of the repeated vertex encoding can be ensured.

That is to say, in the process of encoding the connection relationship of the three-dimensional mesh, for the three vertices of each triangular patch, based on the repeated vertex information, query whether there are repeated vertices; if there are repeated vertices, perform a query on the repeated vertices. Identification, after completing the encoding of the connection relationship, use the entropy encoder to encode the repeated vertex identification into a repeated point identification code stream.

It should also be noted that this application also needs to encode texture coordinates (UV coordinates). It should be noted that UV coordinates are information that describes the texture of the vertices of a three-dimensional grid. The three-dimensional grid first projects the surface texture into two dimensions. , forming a two-dimensional texture map. The UV coordinates represent the position of the three-dimensional vertex texture in the two-dimensional texture map, and correspond to the geometric information one-to-one. Optionally, the implementation process of UV coordinate encoding in an embodiment of the present application is as follows :

The encoding end restores the second geometric information and the repeated vertices in the second connection relationship according to the order of the repeated vertices in the reordered texture coordinate information;

The encoding end traverses the second connection relationship of the restored repeated vertices and records the traversal order;

The encoding end encodes the texture coordinates of the vertices according to the traversal order.

It should be noted that in the above process, the connection relationship is processed to remove duplicate vertices, and the texture coordinate encoding is based on the connection relationship and geometric information containing repeated vertices. Therefore, the second connection relationship needs to be restored first. and repeated vertices in geometric information to ensure the accuracy of texture coordinate encoding.

It should be noted that in the embodiment of the present application, the texture coordinate information after reordering is used to restore the third geometric information obtained by restoring the repeated vertices in the second geometric information and the second connection relationship (after performing repeated vertex restoration on the second geometric information) Corresponding geometric information) and the third connection relationship (that is, the corresponding connection relationship after repeated vertex restoration of the second connection relationship), encode the texture coordinates of the vertices.

The specific implementation process includes:

Step S41, the encoding end determines a target triangle according to the third connection relationship, and the target triangle is composed of the first side and the vertex to be encoded;

Optionally, in the embodiment of this application, the optional implementation method of this step is:

The encoding end selects a first edge from an edge set, which is a set of edges of a triangle constructed by a third connection relationship; the encoding end determines a target triangle based on the first edge.

Optionally, before the encoding end selects the first edge in the edge set, it also includes:

The encoding end selects an initial triangle according to the third connection relationship; the encoding end encodes the texture coordinates of the three vertices of the initial triangle, and adds the three edges of the initial triangle to the edge set.

Typically, this initial triangle is the first triangle in the connection. For the initial triangle, in the embodiment of this application, the vertices are not predicted, but the texture coordinates are directly encoded, and the texture coordinates of each vertex of the initial triangle are encoded (it should be noted that the texture coordinates refer to the target three-dimensional mesh through the After directly obtaining the original texture coordinates), each edge of the initial triangle is added to the edge set to form an initial edge set, and then the subsequent vertices are predicted based on the initial edge set.

Step S42: The encoding end performs texture coordinate prediction on the vertices to be encoded based on the reconstructed geometric information corresponding to the three vertices of the target triangle and the real texture coordinates of the vertices on the first side, and obtains the to-be-encoded vertices. The predicted texture coordinates of the vertex;

Step S43: The encoding end encodes the texture coordinates of the vertex to be encoded based on the difference between the real texture coordinates of the vertex to be encoded and the predicted texture coordinates.

It should be noted that after obtaining the predicted texture coordinates of the vertex to be encoded, the difference between the predicted texture coordinates and the real texture coordinates can be obtained, and then encoding is performed based on the difference. Optionally, the difference can be encoded directly. , you can also process the difference first, and then encode the processed difference. For example, this processing can be normalization processing; by encoding based on the difference, the encoding of the vertices to be encoded can be achieved, which can reduce the texture The number of bits to encode the coordinates.

It should be noted that the difference value described in the embodiment of the present application can be obtained by subtracting the predicted texture coordinates from the real texture coordinates, or by subtracting the predicted texture coordinates from the predicted texture coordinates. The specific method is as long as The encoding end and the decoding end only need to have the same understanding. The difference in the embodiment of this application may also be called a residual.

Optionally, in an embodiment of the present application, the encoding end performs coding on the vertices to be encoded based on the reconstructed geometric information corresponding to the three vertices of the target triangle and the real texture coordinates of the vertices on the first side. Texture coordinate prediction, the implementation method of obtaining the predicted texture coordinates of the vertex to be encoded, includes:

Step S51: The encoding end obtains the texture coordinates of the projection point of the vertex to be encoded on the first side based on the geometric information corresponding to each vertex of the target triangle and the real texture coordinates of the vertex on the first side. ;

It should be noted that, as shown in Figure 9, edge NP is an edge selected from the edge set, which can be regarded as the first edge mentioned above. Vertex N and vertex P are respectively the two vertices of the first edge. C is the vertex to be encoded. Vertex N, vertex P and vertex C form the above target triangle. Point X is the projection of vertex C on the NP edge. Vertex O is the encoded point, and vertex O, vertex N and vertex The triangle formed by P shares NP sides with the triangle formed by vertex N, vertex P and vertex C. Based on Figure 9 above, optionally, the specific method of obtaining the texture coordinates of the projection point of the vertex on the first side mentioned in the embodiment of this application is:

According to the formula: Get the texture coordinates of the projection point of the vertex on the first edge;

_Wherein _, is a vector from the texture coordinates of the vertex N on the first side of the target triangle to the projection point X of the vertex C to be encoded on the first side; is a vector of reconstructed geometric information corresponding to vertex N and vertex P on the first edge; is the vector of reconstructed geometric information corresponding to the vertex N on the first edge to the vertex C to be encoded; is a vector of reconstructed geometric information corresponding to the vertex N on the first edge to the projection point X of the vertex C to be encoded on the first edge.

Step S52: The encoding end obtains the predicted texture coordinates of the vertex to be encoded based on the texture coordinates of the projection point;

Optionally, the predicted texture coordinates of the vertices to be encoded include:

According to the formula: Get the predicted texture coordinates of the vertex to be encoded;

Where, Pred _C is the predicted texture coordinate of the vertex C to be encoded, is the first vector value of the vector of the texture coordinates of the vertex C to be encoded and the first side corresponding to the first vertex O, and O _uv is the first vertex corresponding to the first side of the target triangle. Texture coordinates, the first vertex O is the opposite vertex of the first side of the first triangle (that is, the vertex in the first triangle opposite to the first side), the first triangle and the target triangle have a common first side; is a vector from the projection point X of the vertex C to be encoded on the first edge to the texture coordinates of the vertex C to be encoded; The second vector value is the vector of the texture coordinates of the vertex C to be encoded and the first edge corresponding to the first vertex O.

It should be noted that based on the above process, the predicted texture coordinates of the point to be encoded are obtained, and based on the predicted texture coordinates, the encoding point to be encoded can be implemented; optionally, the encoding point to be encoded is based on an edge in the edge set After encoding, the encoding end adds the second edge in the target triangle to the edge set and deletes the first edge from the edge set. The second edge is not included in the target triangle. on the edges in the edge set, in order to achieve Updates to edge collections.

It should be noted that in the embodiment of the present application, the residuals of the vertices to be encoded can be obtained while encoding, or all the residuals can be obtained first, and then the residuals can be encoded uniformly.

It should also be noted that, in order to ensure the consistency of texture coordinates, in one embodiment of the present application, the encoding end needs to adjust the values and arrangement order of some texture coordinates based on the reconstructed first geometric information.

It should be noted that an embodiment of the present application also includes encoding of texture maps, specifically: for texture maps, a video encoder can usually be used directly to encode frame-by-frame texture maps, such as using high-efficiency video coding ( Encoders such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) form texture image sub-streams. The video encoder here can choose any video encoder.

It should be noted that after obtaining the sub-streams of each channel, the sub-streams are mixed to form an output trellis-encoded code stream.

In summary, it can be seen that for the encoding end, the geometric information, connection relationships, UV coordinates and texture maps of the mesh are encoded respectively. The main process is as follows:

1) In lossy mode, perform optional simplification operations on the three-dimensional mesh, that is, reduce the number of mesh vertices and edges while maintaining the mesh structure as much as possible, thereby reducing the data volume of the three-dimensional mesh.

2) For the simplified mesh, its geometric information is quantified to reduce the distance between vertices, which is beneficial to subsequent video-based geometric information compression.

It should be noted that quantization may obtain low-precision geometric information, high-precision geometric information and supplementary point information. Among them, low-precision geometric information is the three-dimensional coordinates after quantization; high-precision geometric information is the geometric information lost by the low-precision grid after quantization; supplementary points refer to points generated during the quantization process that need to be processed separately, such as overlapping coordinate positions. Repeat point and so on. For the low-precision geometric information generated by quantization, a projection method similar to V-PCC is used to project it to a two-dimensional plane, and then perform video compression. High-precision geometric information and supplementary points can be encoded using the special point processing method in V-PCC, such as using the raw patch method for compression. The obtained sub-code stream is mixed, and is prepared to be further mixed with the connection relationship code stream and other code streams to obtain the final output code stream. The representation and encoding of information of different precisions and supplementary points here take into account both lossless and lossy modes. In particular, lossless compression can be guaranteed while reducing encoding bit overhead.

3) Decode the encoded geometric information code stream to reconstruct the geometric information as auxiliary information for connection relationship encoding, and then encode the updated connection information after updating the connection relationship of the original grid. Finally, the obtained connection relationship code stream is mixed with the code stream of geometric information and other attribute information that needs to be encoded.

4) Use the reconstructed geometric information and UV coordinate information to reconstruct the UV coordinates, and encode the UV coordinates to form a UV coordinate sub-stream. The encoding method here can use predictive coding, that is, predicting the position of the UV coordinates to be encoded and then encoding the residual between the predicted coordinates and the real coordinates to save bit overhead (the number of bits in the residual is smaller than the number of bits in the original UV coordinates).

5) Video encode the texture map to form a texture map sub-stream.

6) Mix the obtained sub-streams into the output code stream of the encoder.

For example, the coding framework of the video-based three-dimensional grid in the embodiment of the present application is shown in Figure 10. The overall coding process is: first, before quantization, you can choose whether to perform sampling simplification on the three-dimensional grid; then, perform sampling on the three-dimensional grid. Quantification, which may produce three parts: low-precision geometric information, high-precision geometric information and supplementary point information; for low-precision geometric information, projection is used to divide patches and arrange patches to generate patch sequence compression information (patch division information) , placeholder map and low-precision geometric map; for possible high-precision geometric information, raw patch arrangement can be used to generate high-precision geometric map; for possible supplementary points, the geometric information of supplementary points can be divided into low-precision For the partial and high-precision parts, the raw patches are arranged separately and separately encoded into a code stream, or the raw patches are added to the geometric diagram; then, the patch sequence compression information, placeholder map, and geometric diagram are encoded. For the connection relationship, the encoded geometric information code stream is decoded to reconstruct the geometric information as auxiliary information for connection relationship encoding. After updating the connection relationship of the original grid, the updated connection information is encoded to obtain the connection relationship sub-stream. For UV coordinates, the reconstructed geometric information and UV coordinate information are used to reconstruct the UV coordinates, and the UV coordinates are encoded to form a UV coordinate sub-stream. In some cases, such as when the texture map fragmentation is relatively trivial, you can choose to regenerate the UV coordinates with more complete fragmentation and encode them. For texture maps, the texture map can be directly video-encoded to form a texture map sub-stream. Finally, multiple sub-streams are mixed to obtain the final output stream.

It should be noted that the embodiment of the present application provides a coding framework based on a three-dimensional grid of video. For the decoding end, it only needs to be decoded according to the inverse process of the encoding end; in the implementation of the present application, it is particularly targeted at the video containing repeated geometric information. The grid of vertices is individually and concisely marked and coded. In the process of coding connection relationships, because the coding of repeated vertices is removed, the coding of the three-dimensional grid can be made more efficient.

For the encoding method provided by the embodiment of the present application, the execution subject may be an encoding device. In the embodiment of the present application, the encoding device performing the encoding method is taken as an example to illustrate the encoding device provided by the embodiment of the present application.

As shown in Figure 11, this embodiment of the present application provides an encoding device 1100, which includes:

The first acquisition module 1101 is used to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain the reconstructed first geometric information;

The second acquisition module 1102 is used to filter repeated vertices on the first geometric information and obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex is The vertex is one of multiple vertices with the same position coordinates;

The third acquisition module 1103 is configured to reorder the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtain the reordered second connection relationships;

The first encoding module 1104 is used to encode the second connection relationship.

Optionally, the method of obtaining repeated vertex information includes:

Obtain the repeated vertex in the first geometric information, record the index of the repeated vertex to form the repeated vertex information;

Optionally, the third acquisition module 1103 includes:

The first acquisition unit is used to re-acquire the vertex texture coordinates of the target three-dimensional mesh according to the index of the repeated vertex. Arrange to obtain the texture coordinate information after reordering;

The second acquisition unit is configured to adjust the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtain the second connection relationship after the reordering.

Optionally, the first acquisition unit is used for:

According to the order in which the repeated vertices appear in the first geometric information, the texture coordinates corresponding to the repeated vertices are sequentially moved to the end of the list corresponding to the texture coordinates to form reordered texture coordinate information.

Optionally, the second acquisition unit is used for:

Modify the texture coordinate index part of the first connection relationship of the target three-dimensional grid according to the texture coordinate information after reordering;

Traverse the geometric index part of the first connection relationship of the target three-dimensional grid, replace the index of the repeated vertex of the first connection relationship of the target three-dimensional grid with the index of the first vertex according to the repeated vertex information, and obtain the repeated vertex index. The second connection relationship after sorting.

Optionally, the device also includes:

A first query module, configured to query whether there are duplicate vertices based on the duplicate vertex information for the three vertices of each triangular patch during the process of encoding the second connection relationship;

An identification module, used to identify duplicate vertices when they exist;

The fourth acquisition module is used to perform encoding according to the identifier of the repeated vertex after completing the encoding of the second connection relationship, and obtain the code stream of the repeated vertex identifier.

Optionally, after the first acquisition module 1101 decodes and reconstructs the encoded geometric information of the target three-dimensional mesh and obtains the reconstructed first geometric information, it also includes:

An adjustment module, configured to adjust the values and arrangement order of some texture coordinates according to the reconstructed first geometric information.

Optionally, the device also includes:

A first recovery module, configured to restore the second geometric information and the repeated vertices in the second connection relationship according to the order of the repeated vertices in the reordered texture coordinate information, where the second geometric information does not include repeated vertices. Mesh geometry information;

The first recording module is used to traverse the second connection relationship to restore repeated vertices and record the traversal order;

The second encoding module is used to encode the texture coordinates of the vertices according to the traversal order.

Optionally, before the third acquisition module 1103 reorders the first connection relationships of the target three-dimensional mesh according to the repeated vertex information and obtains the reordered second connection relationship, it also includes:

An adjustment module, configured to adjust the portion where the first connection relationship of the target three-dimensional grid changes according to the reconstructed first geometric information.

This device embodiment corresponds to the above-mentioned encoding method embodiment. Each implementation process and implementation manner of the above-mentioned method embodiment can be applied to this device embodiment, and can achieve the same technical effect.

Embodiments of the present application also provide an encoding device, including a processor and a communication interface, wherein the processor is used to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain the reconstructed first geometric information; Right Perform repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex is among the multiple vertices with the same position coordinates. one of the vertices; perform reordering of the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtain the reordered second connection relationship; and encode the second connection relationship.

Optionally, the processor is used for:

The encoding end rearranges the vertex texture coordinates of the target three-dimensional grid according to the index of the repeated vertex, and obtains the reordered texture coordinate information;

The encoding end adjusts the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtains the second connection relationship after the reordering.

Optionally, the processor is used for:

The encoding end sequentially moves the texture coordinates corresponding to the repeated vertices to the end of the list corresponding to the texture coordinates according to the order in which the repeated vertices appear in the first geometric information, to form reordered texture coordinate information.

Optionally, the processor is used for:

The encoding end modifies the texture coordinate index part of the first connection relationship of the target three-dimensional grid according to the texture coordinate information after reordering;

The encoding end traverses the geometric index part of the first connection relationship of the target three-dimensional grid, and replaces the index of the repeated vertex of the first connection relationship of the target three-dimensional grid with the index of the first vertex according to the repeated vertex information. Index to obtain the second connection relationship after reordering.

Optionally, the processor is also used to:

The encoding end adjusts the values and arrangement order of some texture coordinates according to the reconstructed first geometric information.

Optionally, the processor is also used to:

The encoding end restores the second geometric information and the repeated vertices in the second connection relationship according to the order of the repeated vertices in the reordered texture coordinate information, and the second geometric information is a mesh geometry that does not include repeated vertices. information;

Optionally, the processor is also used to:

The encoding end adjusts the changed portion of the first connection relationship of the target three-dimensional grid according to the reconstructed first geometric information.

Specifically, this embodiment of the present application also provides an encoding device. As shown in Figure 12, the encoding device 1200 includes: a processor 1201, a network interface 1202, and a memory 1203. The network interface 1202 is, for example, a common public radio interface (CPRI).

Specifically, the encoding device 1200 in the embodiment of the present application also includes: instructions or programs stored in the memory 1203 and executable on the processor 1201. The processor 1201 calls the instructions or programs in the memory 1203 to execute the modules shown in Figure 11 The implementation method and achieve the same technical effect will not be repeated here to avoid repetition.

Corresponding to the implementation process of the encoding end, as shown in Figure 13, embodiments of the present application also provide a decoding method, including:

Step 1301: The decoder decodes and reconstructs the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

Step 1302: The decoder performs repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex Is one of the vertices among multiple vertices with the same position coordinates;

Step 1303: The decoding end decodes the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the method of obtaining repeated vertex information includes:

The decoding end obtains the repeated vertices in the first geometric information, and records the index of the repeated vertices to form repeated vertex information;

Optionally, the method also includes:

During the process of decoding the connection relationship code stream, the decoding end queries whether there are duplicate vertices for each triangle patch based on the duplicate vertex information;

When a duplicate vertex exists, the decoding end decodes the index of the duplicate vertex from the duplicate identification code stream.

Optionally, the method also includes:

The decoding end restores the repeated vertices according to the second geometric information and the decoded connection relationship according to the index of the repeated vertex, where the second geometric information is mesh geometry information excluding repeated vertices;

The decoding end traverses the connection relationship of the restored repeated vertices and records the traversal order;

The decoding end decodes the texture coordinate code stream of the vertex according to the traversal order.

Optionally, the video-based three-dimensional grid geometric information decoding process in the embodiment of this application includes: decomposing the code stream into a patch information sub-stream, a placeholder image code stream, and a geometric image code stream; and then decoding these three sub-streams respectively. Code stream, obtain patch information, placeholder map and geometric map; finally, use patch information, placeholder map and geometric map to reconstruct geometric information. Specifically, as shown in Figure 14, the most critical thing is the specific process of geometric information reconstruction:

Step S111, obtain the 2D patch;

It should be noted that obtaining a 2D patch refers to using the patch information to segment the placeholder information and depth information of the 2D patch from the placeholder map and geometric map. The patch information contains the position and size of each 2D patch's bounding box in the placeholder map and low-precision geometric map. The placeholder information and the 2D patch's placeholder information can be directly obtained using the patch information, placeholder map, and low-precision geometric map. Low precision geometric information. For high-precision geometric information, the vertex distribution of the low-precision geometric graph is used to correspond the high-precision geometric information arrangement values in the high-precision geometric information area to the vertices of the low-precision geometric graph, and then xyz is separated from them according to the preset arrangement rules used by the encoding end. Three-dimensional geometric information, thereby obtaining high-precision geometric information. For the geometric information of supplementary points, the low-precision geometric information and high-precision geometric information of supplementary points can be obtained by directly decoding the low-precision raw patch and high-precision raw patch of supplementary points.

Step S112, reconstruct the 3D patch;

It should be noted that reconstructing a 3D patch refers to using the placeholder information and low-precision geometric information in the 2D patch to reconstruct the vertices in the 2D patch into a low-precision 3D patch. The placeholder information of a 2D patch contains the position of the vertex relative to the coordinate origin in the local coordinate system of the patch projection plane, and the depth information contains the depth value of the vertex in the normal direction of the projection plane. Therefore, the 2D patch can be reconstructed into a low-precision 3D patch in the local coordinate system using the occupancy information and depth information.

Step S113, reconstruct the low-precision geometric model;

It should be noted that reconstructing a low-precision geometric model refers to using the reconstructed low-precision 3D patch to reconstruct the entire low-precision three-dimensional geometric model. The patch information contains the conversion relationship of the 3D patch from the local coordinate system to the global coordinate system of the three-dimensional geometric model. Using the coordinate conversion relationship to convert all 3D patches to the global coordinate system, a low-precision three-dimensional geometric model is obtained. In addition, for the supplementary points, the geometric information in the low-precision raw patch is directly used to obtain the low-precision coordinate values of the supplementary points in the global coordinate system, thereby obtaining a complete low-precision three-dimensional geometric model.

Step S114, reconstruct the high-precision geometric model;

It should be noted that reconstructing a high-precision geometric model refers to the process of using high-precision geometric information to reconstruct a high-precision geometric model based on a low-precision geometric model. In the process of obtaining the 2D patch, high-precision geometric information and low-precision geometric information are mapped, and the high-precision three-dimensional coordinates of the vertex can be reconstructed based on the high-precision geometric information and low-precision geometric information of the vertex. According to the requirements of the application, you can choose to reconstruct the high-precision three-dimensional coordinates of all vertices or reconstruct the high-precision three-dimensional coordinates of some vertices. The calculation process of high-precision three-dimensional coordinates (x _r , y _r , z _r ) is as shown in Formula 38 to Formula 40:

Formula 38: x _r =f ₃ (x _l ,x _h ,QP _x );

Formula 39: y _r =f ₃ (y _l ,y _h ,QP _y );

Formula 40: z _r = f ₃ (z _l ,z _h ,QP _z );

The f ₃ function is a reconstruction function. The calculation process of the reconstruction function corresponds to the calculation process of the quantization function at the encoding end, and there are many ways to implement it. If the f ₁ function adopts the implementation method of Formula 20 to Formula 25, then the reconstruction function is implemented as shown in Formula 41 to Formula 43:

Formula 41: x _r =x _l *QP _x +x _h ;

Formula 42: y _r =y _l *QP _y +y _h ;

Formula 43: z _r ＝z _l *QP _z +z _h ;

If the f ₁ function adopts the implementation method of Formula 26 to Formula 31, then the reconstruction function is implemented as shown in Formula 44 to Formula 46:

Formula 44: x _r = (x _l ＜＜log ₂ QP _x )|x _h ;

Formula 45: y _r = (y _l ＜＜log ₂ QP _y )|y _h ;

Formula 46: z _r = (z _l ＜＜log ₂ QP _z )|z _h .

It should be noted that when decoding the connection relationship in the embodiment of the present application, it is necessary to first find the repeated vertices in the reconstructed geometric information, remove the repeated vertices, and record the vertex index of the repeated points to form repeated vertex information. Specifically, the process of decoding the connection relationship may specifically include:

(1) Decode the vertices of the initial triangle and store the edges of the initial triangle into the edge set. Edge τ is taken from the edge set according to the criteria used in the encoder. If the codeword to be decoded is a vertex index, the vertex is directly decoded and used as the pair vertex v. Use the vertex v and the side τ to form a newly decoded triangle, and add the two sides of the triangle except the side τ to the edge set, and remove the edge τ from the set according to a certain rule, such as moving according to the criteria at the top of the queue. Remove edge τ. Take out the next edge according to a certain rule and continue to decode the triangle adjacent to this edge. For example, you can take out the edge at the top of the queue as the rule.

(2) If the codeword to be decoded is not a vertex index, decode and determine the spatial range of the vertex v. For example, to decode the radii of two concentric spheres {R _min , R _max } at the midpoint of τ, traverse all the vertices within the range between the concentric spheres, and filter out the new triangles formed with the side τ that have an angle smaller than a certain angle with the decoded triangle. All vertices.

(3) For the remaining vertices within the space range where vertex v is located, sort the vertices according to the same sorting criteria as the encoding end. Decode the sequence number corresponding to the pair of vertices v in the triangle to be decoded, and look up the table to obtain the pair of vertices v, and construct the decoding triangle. Add the two edges of the new decoded triangle except the edge τ to the edge set, and remove the edge τ in the set according to a certain rule, such as removing the edge τ according to the criterion at the top of the queue. Take out the next edge according to a certain rule and continue to decode the triangle adjacent to this edge. For example, you can take out the edge at the top of the queue as the rule.

The decoding process is iterated for each slice's code stream until all triangles in each slice are decoded. Finally the pieces are merged into a complete mesh.

It should be noted that during the process of decoding the connection relationship, for each triangular patch, according to the repeated point comparison table, if the vertex of the current triangular patch is a repeated vertex, the current repeated vertex index is decoded from the repeated identification code stream. Specifically, first decode the repeated vertex identifier and put it into an array. Then based on the repeated vertex information, if the current vertex is a repeated vertex, the repeated vertex identifier is removed from the array to form a repeated vertex index.

It should be noted that the decoding process of the UV coordinates of the three-dimensional grid is the inverse process of the encoding process. In a specific embodiment of the present application, the coordinate prediction residual is first entropy decoded. Then restore the repeated vertices in the geometric information and connectivity relationship, iterate the connection relationship and record the connection order of the points, and predict the UV coordinates of the vertices based on this order. The prediction process uses the projection mapping relationship between the three-dimensional and two-dimensional triangles, and predicts the current point based on the UV coordinates of the decoded point according to the encoding-side prediction rules to obtain the predicted UV coordinates. Add the predicted coordinates to the residual value decoded by entropy to obtain the UV coordinate position to be decoded.

It should be noted here that the UV coordinates of the initial triangle do not use predictive encoding, but directly encode their The UV coordinate value, after the decoder decodes the UV coordinates of the triangle, is used as the initial triangle to start traversing and decoding the UV coordinates of other triangle vertices.

For the decoding of the texture map, in a specific embodiment of the present application, a video decoder is directly used to decode the texture map, and a frame-by-frame texture map can be obtained. The file format of the texture map is not emphasized here. The format can be jpg, png. wait.

In summary, it can be seen that the three-dimensional grid decoding process can be divided into:

1. After destreaming, decode each sub-stream. Among them, use the video decoder to decode the occupancy map, geometry map, and texture map. Use the decoder corresponding to the encoding method of the encoding end to use the connection relationship and UV coordinate sub-stream. to decode.

2. For geometric information, first decompose the code stream into patch information sub-stream, placeholder image code stream and geometric image code stream; then, decode these three sub-code streams respectively to obtain patch information, placeholder image and geometric image; Finally, the patch information, placeholder map and geometry map are used to reconstruct the geometric information.

3. For the connection relationship, decode the connection relationship sub-stream and reconstruct the connection relationship.

4. For UV coordinates, decode the connection relationship sub-stream and reconstruct the UV coordinates.

5. For the texture map, decode the texture map sub-stream to obtain frame-by-frame texture maps.

It should be noted that the decoding process in the embodiment of the present application is the inverse process of encoding. The decoding block diagram is shown in Figure 15. First, the code stream is decomposed into a patch information sub-stream, a placeholder map sub-stream, and a geometric map sub-stream. Stream, connect the relationship sub-stream, UV coordinate sub-stream, and texture map sub-stream and decode them respectively; use placeholder map and low-precision geometric map to reconstruct the geometric information of low-precision mesh, use placeholder map, low-precision geometric map Geometric diagrams and high-precision geometric diagrams can reconstruct the geometric information of high-precision meshes; use the connection relationship sub-stream to decode the connection relationship of the grid; decode the UV coordinates of the grid from the UV coordinate sub-stream; and decode the UV coordinates of the grid from the texture map sub-stream. The frame-by-frame texture map is decoded from the code stream; finally, the three-dimensional grid is reconstructed using the reconstructed geometric information, connection information, UV coordinate information, texture map and other information.

It should be noted that the embodiment of the present application is a method embodiment of the opposite end corresponding to the embodiment of the above encoding method. The decoding process is the inverse process of encoding. All the above implementation methods on the encoding side are applicable to the embodiment of the decoding end. The same technical effect can also be achieved, which will not be described again here.

As shown in Figure 16, this embodiment of the present application also provides a decoding device 1600, which includes:

The reconstruction module 1601 is used to decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

The fifth acquisition module 1602 is used to perform repeated vertex screening on the first geometric information and obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex is The vertex is one of multiple vertices with the same position coordinates;

The decoding module 1603 is configured to decode the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the method of obtaining repeated vertex information includes:

Optionally, the device also includes:

The second query module is used to query whether there are duplicate vertices for each triangle patch based on the duplicate vertex information during the process of decoding the connection relationship code stream;

The second decoding module is used for the decoding end to decode the index of the repeated vertex from the repeated identification code stream when there is a repeated vertex.

Optionally, the device also includes:

A second recovery module, configured to recover the second geometric information and the decoded connection relationship according to the index of the repeated vertex, where the second geometric information is mesh geometry information excluding repeated vertices;

The second recording module is used to traverse the connection relationships of restored duplicate vertices and record the traversal order;

The third decoding module is used to decode the texture coordinate code stream of the vertices according to the traversal order.

It should be noted that this device embodiment is a device corresponding to the above-mentioned method. All implementation methods in the above-mentioned method embodiment are applicable to this device embodiment and can achieve the same technical effect, which will not be described again here.

Preferably, the embodiment of the present application also provides a decoding device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the above-mentioned decoding device is implemented. Each process of the decoding method embodiment can achieve the same technical effect. To avoid repetition, it will not be described again here.

Embodiments of the present application also provide a readable storage medium. Programs or instructions are stored on the computer-readable storage medium. When the program or instructions are executed by a processor, each process of the above-mentioned decoding method embodiment is implemented, and the same process can be achieved. To avoid repetition, the technical effects will not be repeated here.

Among them, the computer-readable storage medium is such as read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

An embodiment of the present application also provides a decoding device, including a processor and a communication interface, wherein the processor is used to decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid; The geometric information is filtered for repeated vertices to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex is one of the multiple vertices with the same position coordinates. Vertex: decode the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the processor is used for:

In the process of decoding the connection relationship code stream, for each triangle patch, based on the repeated vertex information, query whether there are repeated vertices;

In the case where duplicate vertices exist, the index of the duplicate vertex is decoded from the duplicate identification code stream.

Optionally, the processor is used for:

Restoring the repeated vertices according to the second geometric information and the decoded connection relationship according to the index of the repeated vertices, where the second geometric information is mesh geometry information excluding repeated vertices;

Traverse the connection relationships of restored duplicate vertices and record the traversal order;

According to the traversal order, the texture coordinate code stream of the vertex is decoded.

This decoding device embodiment corresponds to the above-mentioned decoding method embodiment. Each implementation process and implementation manner of the above-mentioned method embodiment can be applied to this decoding device embodiment, and can achieve the same technical effect.

Specifically, the embodiment of the present application also provides a decoding device. Specifically, the structure of the decoding device is shown in Figure 12, which will not be described again here. Specifically, the decoding device in the embodiment of the present application also includes: instructions or programs stored in the memory and executable on the processor. The processor calls the instructions or programs in the memory to execute the method executed by each module shown in Figure 16, and To achieve the same technical effect, to avoid repetition, we will not repeat them here.

Embodiments of the present application also provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the program or instructions are executed by a processor, each process of the above decoding method embodiment is implemented, and the same process can be achieved. To avoid repetition, the technical effects will not be repeated here.

Wherein, the processor is the processor in the decoding device described in the above embodiment. The readable storage medium includes computer readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc.

Optionally, as shown in Figure 17, this embodiment of the present application also provides a communication device 1700, including a processor 1701 and a memory 1702. The memory 1702 stores programs or instructions that can be run on the processor 1701, for example , when the communication device 1700 is a coding device, when the program or instruction is executed by the processor 1701, each step of the above coding method embodiment is implemented, and the same technical effect can be achieved. When the communication device 1700 is a decoding device, when the program or instruction is executed by the processor 1701, each step of the above decoding method embodiment is implemented, and the same technical effect can be achieved. To avoid duplication, the details are not repeated here.

An embodiment of the present application further provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the above encoding method or decoding method. Each process in the example can achieve the same technical effect. To avoid repetition, we will not repeat it here.

It should be understood that the chips mentioned in the embodiments of this application may also be called system-on-chip, system-on-a-chip, system-on-chip or system-on-chip, etc.

Embodiments of the present application further provide a computer program/program product. The computer program/program product is stored in a storage medium. The computer program/program product is executed by at least one processor to implement the above encoding method or decoding method. Each process of the embodiment can achieve the same technical effect, so to avoid repetition, it will not be described again here.

Embodiments of the present application also provide a communication system, which at least includes: an encoding device and a decoding device. The encoding device can be used to perform the steps of the encoding method as described above. The decoding device can be used to perform the decoding method as described above. A step of. And can achieve the same technical effect. To avoid repetition, they will not be described again here.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element qualified by the statement "includes a..." does not exclude There are also other identical elements in a process, method, article, or device that includes that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions may be performed, for example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk , CD), including several instructions to cause a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Inspired by this application, many forms can be made without departing from the purpose of this application and the scope protected by the claims, all of which fall within the protection of this application.

Claims

A coding method that includes:

The encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid, and obtains the reconstructed first geometric information;

The encoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates, and the first vertex is the position coordinate. One of the same vertices;

The encoding end reorders the first connection relationships of the target three-dimensional grid based on the repeated vertex information, and obtains the reordered second connection relationships;

The encoding end encodes the second connection relationship.
The method according to claim 1, wherein the method of obtaining repeated vertex information includes:

The encoding end obtains the repeated vertices in the first geometric information, and records the index of the repeated vertices to form repeated vertex information;

Wherein, the repeated vertex information includes the index of the repeated vertex and the texture coordinate index of the first vertex and the repeated vertex.
The method according to claim 1, wherein the encoding end reorders the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtains the reordered second connection relationship, including:

The encoding end rearranges the vertex texture coordinates of the target three-dimensional grid according to the index of the repeated vertex, and obtains the reordered texture coordinate information;

The encoding end adjusts the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtains the second connection relationship after the reordering.
The method according to claim 3, wherein the encoding end rearranges the vertex texture coordinates of the target three-dimensional grid according to the index of the repeated vertex, and obtains the texture coordinate information after the reordering, including:

The encoding end sequentially moves the texture coordinates corresponding to the repeated vertices to the end of the list corresponding to the texture coordinates according to the order in which the repeated vertices appear in the first geometric information, to form reordered texture coordinate information.
The method according to claim 3, wherein the encoding end adjusts the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtains the reordered texture coordinate information. The second connection relationship includes:

The encoding end modifies the texture coordinate index part of the first connection relationship of the target three-dimensional grid according to the texture coordinate information after reordering;

The encoding end traverses the geometric index part of the first connection relationship of the target three-dimensional grid, and replaces the index of the repeated vertex of the first connection relationship of the target three-dimensional grid with the index of the first vertex according to the repeated vertex information. Index to obtain the second connection relationship after reordering.
The method of claim 1, further comprising:

In the process of encoding the second connection relationship, the encoding end queries the three vertices of each triangular patch to see whether there are repeated vertices based on the repeated vertex information;

When a duplicate vertex exists, the encoding end identifies the duplicate vertex;

After completing the encoding of the second connection relationship, the encoding end performs encoding according to the identifier of the repeated vertex, and obtains the code stream of the repeated vertex identifier.
The method according to claim 1, wherein, at the encoding end, the geometric information of the encoded target three-dimensional grid is decoded and reconstructed, and after the reconstructed first geometric information is obtained, the method further includes:

The encoding end adjusts the values and arrangement order of some texture coordinates according to the reconstructed first geometric information.
The method of claim 3, further comprising:

The encoding end restores the second geometric information and the repeated vertices in the second connection relationship according to the order of the repeated vertices in the reordered texture coordinate information, and the second geometric information is a mesh geometry that does not include repeated vertices. information;

The encoding end traverses the second connection relationship of the restored repeated vertices and records the traversal order;

The encoding end encodes the texture coordinates of the vertices according to the traversal order.
The method according to claim 1, wherein before the encoding end reorders the first connection relationships of the target three-dimensional mesh according to the repeated vertex information and obtains the reordered second connection relationship, Also includes:

The encoding end adjusts the changed portion of the first connection relationship of the target three-dimensional grid according to the reconstructed first geometric information.
A decoding method including:

The decoding end decodes and reconstructs the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

The decoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates, and the first vertex is the position coordinate. One of the same vertices;

The decoding end decodes the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.
The method according to claim 10, wherein the method of obtaining repeated vertex information includes:

The decoding end obtains the repeated vertices in the first geometric information, and records the index of the repeated vertices to form repeated vertex information;

Wherein, the repeated vertex information includes the index of the repeated vertex and the texture coordinate index of the first vertex and the repeated vertex.
The method of claim 10, further comprising:

During the process of decoding the connection relationship code stream, the decoding end queries whether there are duplicate vertices for each triangle patch based on the duplicate vertex information;

When a duplicate vertex exists, the decoding end decodes the index of the duplicate vertex from the duplicate identification code stream.
The method of claim 10, further comprising:

The decoding end restores the repeated vertices according to the second geometric information and the decoded connection relationship according to the index of the repeated vertex, where the second geometric information is mesh geometry information excluding repeated vertices;

The decoding end traverses the connection relationship of the restored repeated vertices and records the traversal order;

The decoding end decodes the texture coordinate code stream of the vertex according to the traversal order.
An encoding device comprising:

The first acquisition module is used to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain the reconstructed first geometric information;

The second acquisition module is used to filter the repeated vertices of the first geometric information and obtain the repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex Is one of the vertices among multiple vertices with the same position coordinates;

A third acquisition module, configured to reorder the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtain the reordered second connection relationships;

The first encoding module is used to encode the second connection relationship.
The device according to claim 14, wherein the method of obtaining repeated vertex information includes:

Obtain the repeated vertex in the first geometric information, record the index of the repeated vertex to form the repeated vertex information;

Wherein, the repeated vertex information includes the index of the repeated vertex and the texture coordinate index of the first vertex and the repeated vertex.
The device according to claim 14, wherein the third acquisition module includes:

The first acquisition unit is used to rearrange the vertex texture coordinates of the target three-dimensional grid according to the index of the repeated vertex, and acquire the texture coordinate information after the reordering;

The second acquisition unit is configured to adjust the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtain the second connection relationship after the reordering.
The device according to claim 16, wherein the first acquisition unit is used for:

According to the order in which the repeated vertices appear in the first geometric information, the texture coordinates corresponding to the repeated vertices are sequentially moved to the end of the list corresponding to the texture coordinates to form reordered texture coordinate information.
The device according to claim 16, wherein the second acquisition unit is used for:

Modify the texture coordinate index part of the first connection relationship of the target three-dimensional grid according to the texture coordinate information after reordering;

Traverse the geometric index part of the first connection relationship of the target three-dimensional grid, replace the index of the repeated vertex of the first connection relationship of the target three-dimensional grid with the index of the first vertex according to the repeated vertex information, and obtain the repeated vertex index. The second connection relationship after sorting.
The device of claim 14, further comprising:

A first query module, configured to query whether there are duplicate vertices based on the duplicate vertex information for the three vertices of each triangular patch during the process of encoding the second connection relationship;

An identification module, used to identify duplicate vertices when they exist;

The fourth acquisition module is used to perform encoding according to the identifier of the repeated vertex after completing the encoding of the second connection relationship, and obtain the code stream of the repeated vertex identifier.
The device according to claim 14, wherein after the first acquisition module decodes and reconstructs the geometric information of the encoded target three-dimensional grid and obtains the reconstructed first geometric information, it further includes:

An adjustment module, configured to adjust the values and arrangement order of some texture coordinates according to the reconstructed first geometric information.
The device of claim 16, further comprising:

A first recovery module, configured to restore the second geometric information and the repeated vertices in the second connection relationship according to the order of the repeated vertices in the reordered texture coordinate information, where the second geometric information does not include repeated vertices. Mesh geometry information;

The first recording module is used to traverse the second connection relationship to restore repeated vertices and record the traversal order;

The second encoding module is used to encode the texture coordinates of the vertices according to the traversal order.
The device according to claim 14, wherein the third acquisition module reorders the first connection relationships of the target three-dimensional mesh according to the repeated vertex information, and obtains the reordered second connection relationships. Previously, this also included:

An adjustment module, configured to adjust the portion where the first connection relationship of the target three-dimensional grid changes according to the reconstructed first geometric information.
A coding device, including a processor and a memory, the memory stores a program or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the implementation of any one of claims 1 to 9 is achieved. The steps of the encoding method.
A decoding device including:

The reconstruction module is used to decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

The fifth acquisition module is used to perform repeated vertex screening on the first geometric information and obtain repeated vertex information. The repeated vertices are vertices other than the first vertex among multiple vertices with the same position coordinates. The first vertex Is one of the vertices among multiple vertices with the same position coordinates;

The first decoding module is used to decode the connection relationship code stream of the target three-dimensional grid according to the repeated vertex information.
A decoding device, including a processor and a memory, the memory stores a program or instructions that can be run on the processor, and when the program or instructions are executed by the processor, any one of claims 10 to 13 is implemented. The steps of the decoding method.
A readable storage medium on which programs or instructions are stored. When the programs or instructions are executed by a processor, the steps of the encoding method as claimed in any one of claims 1 to 9 are implemented or as claimed in the claims. The steps of the decoding method described in any one of 10 to 13.