CN116934880A

CN116934880A - Encoding and decoding methods, devices and equipment

Info

Publication number: CN116934880A
Application number: CN202210370125.8A
Authority: CN
Inventors: 邹文杰; 张伟; 杨付正; 吕卓逸
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2022-04-08
Filing date: 2022-04-08
Publication date: 2023-10-24
Also published as: WO2023193709A1

Abstract

The application discloses a coding and decoding method, a device and equipment, and relates to the technical field of coding and decoding. The encoding method comprises the following steps: the encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid to obtain reconstructed first geometric information; the encoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information, wherein the repeated vertices are among multiple vertices with the same position coordinates and are except for a first vertex, and the first vertex is one of the multiple vertices with the same position coordinates; the encoding end reorders the first connection relation of the target three-dimensional grid according to the repeated vertex information, and acquires a reordered second connection relation; the encoding end encodes the second connection relation.

Description

Encoding and decoding methods, devices and equipment

Technical Field

The application belongs to the technical field of encoding and decoding, and particularly relates to an encoding and decoding method, an encoding and decoding device and encoding and decoding equipment.

Background

The three-dimensional grid has the similar characteristics by considering that the vertexes and the point clouds of the three-dimensional grid are a group of irregularly distributed discrete point sets in the space. Therefore, geometric information in a three-dimensional grid can also be compressed using video-based point cloud compression standards. However, the vertexes of the three-dimensional grid have the characteristics of sparse spatial distribution and uneven spatial distribution, and the compression efficiency of the point cloud compression method based on video is low. Furthermore, compression of mesh vertex connection relationships, mesh UV coordinate attributes, and texture maps is not involved in video-based point cloud compression standards. Because of the generation mechanism of the three-dimensional grid, partial geometric repeated points exist, the geometric coordinates of the geometric repeated points are the same, the UV coordinates are different, the geometric repeated points can cause the problem of discontinuity of the three-dimensional grid, the situation that a plurality of sides are adjacent to one triangle only occurs, and the single side needs to consume redundant code words for identification, so that the coding efficiency of the connectivity relation is reduced.

Disclosure of Invention

The embodiment of the application provides a coding and decoding method, a device and equipment, which can solve the problem that the three-dimensional grid generating mechanism in the prior art can cause partial geometric repeated points, the geometric repeated points can cause the discontinuity of the three-dimensional grid, and the coding efficiency of the connectivity relation of the three-dimensional grid is reduced.

In a first aspect, there is provided an encoding method comprising:

the encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid to obtain reconstructed first geometric information;

the encoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information, wherein the repeated vertices are among multiple vertices with the same position coordinates and are except for a first vertex, and the first vertex is one of the multiple vertices with the same position coordinates;

the encoding end reorders the first connection relation of the target three-dimensional grid according to the repeated vertex information, and acquires a reordered second connection relation;

the encoding end encodes the second connection relation.

In a second aspect, there is provided an encoding apparatus comprising:

the first acquisition module is used for decoding and reconstructing the geometric information of the coded target three-dimensional grid to acquire reconstructed first geometric information;

The second acquisition module is used for carrying out repeated vertex screening on the first geometric information to acquire repeated vertex information, wherein the repeated vertexes are among the multiple vertexes with the same position coordinates and are except for a first vertex, and the first vertex is one vertex among the multiple vertexes with the same position coordinates;

the third acquisition module is used for reordering the first connection relation of the target three-dimensional grid according to the repeated vertex information and acquiring a reordered second connection relation;

and the first coding module is used for coding the second connection relation.

In a third aspect, a decoding method is provided, comprising:

the decoding end decodes and rebuilds the first geometric information according to the code stream corresponding to the obtained target three-dimensional grid;

the decoding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information, wherein the repeated vertices are among multiple vertices with the same position coordinates and are except for a first vertex, and the first vertex is one of the multiple vertices with the same position coordinates;

and the decoding end decodes the first connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

In a fourth aspect, there is provided a decoding apparatus including:

the reconstruction module is used for decoding and reconstructing the first geometric information according to the code stream corresponding to the obtained target three-dimensional grid;

a fifth obtaining module, configured to perform repeated vertex screening on the first geometric information, to obtain repeated vertex information, where the repeated vertex is a vertex other than a first vertex in multiple vertices with the same position coordinates, and the first vertex is one vertex in multiple vertices with the same position coordinates;

and the first decoding module is used for decoding the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

In a fifth aspect, there is provided an encoding apparatus comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, performs the steps of the method according to the first aspect.

In a sixth aspect, an encoding device is provided, including a processor and a communication interface, where the processor is configured to decode and reconstruct geometric information of an encoded target three-dimensional grid, and obtain reconstructed first geometric information; the first geometric information is subjected to repeated vertex screening, repeated vertex information is obtained, the repeated vertexes are among the vertexes with the same position coordinates, except for a first vertex, and the first vertex is one vertex among the vertexes with the same position coordinates; according to the repeated vertex information, reordering the first connection relation of the target three-dimensional grid, and acquiring a reordered second connection relation; and encoding the second connection relation.

In a seventh aspect, there is provided a decoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method according to the third aspect.

An eighth aspect provides a decoding device, including a processor and a communication interface, where the processor is configured to decode and reconstruct first geometric information according to a code stream corresponding to an obtained target three-dimensional grid; the first geometric information is subjected to repeated vertex screening, repeated vertex information is obtained, the repeated vertexes are among the vertexes with the same position coordinates, except for a first vertex, and the first vertex is one vertex among the vertexes with the same position coordinates; and decoding the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

In a ninth aspect, there is provided a communication system comprising: an encoding device operable to perform the steps of the method as described in the first aspect and a decoding device operable to perform the steps of the method as described in the third aspect.

In a tenth aspect, there is provided a readable storage medium having stored thereon a program or instructions which when executed by a processor, performs the steps of the method according to the first aspect, or performs the steps of the method according to the third aspect.

In an eleventh aspect, there is provided a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being for running a program or instructions to implement the method according to the first aspect or to implement the method according to the third aspect.

In a twelfth aspect, there is provided a computer program/program product stored in a storage medium, the computer program/program product being executed by at least one processor to implement the steps of the method as described in the first aspect.

In the embodiment of the application, the reconstructed first geometric information is obtained by decoding and reconstructing the geometric information of the coded target three-dimensional grid, repeated vertex screening is carried out on the first geometric information to obtain repeated vertex information, the first connection relation of the target three-dimensional grid is reordered according to the repeated vertex information, the reordered second connection relation is obtained, and the second connection relation is coded; according to the scheme, the repeated vertexes are screened by using the reconstructed geometric information, then the connection relation is reordered based on the repeated vertexes information, and the codes are carried out, so that the problem of discontinuous three-dimensional grids caused by the repeated vertexes can be avoided, and the coding efficiency of the connection relation of the three-dimensional grids is improved.

Drawings

FIG. 1 is a flow chart of an encoding method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a process of merging vertices in mesh simplification;

FIG. 3 is a schematic diagram of a grid-based fine partitioning process;

FIG. 4 is a schematic diagram of eight directions of the Patch arrangement;

FIG. 5 is a schematic diagram of a process for encoding high precision geometric information;

FIG. 6 is a schematic diagram of a raw patch;

FIG. 7 is a diagram of one of the encoded connection relationships in an embodiment of the present application;

FIG. 8 is a diagram illustrating a second code connection relationship in accordance with an embodiment of the present application;

FIG. 9 is a geometric schematic of the prediction principle;

FIG. 10 is a schematic diagram of a video-based three-dimensional grid coding framework;

FIG. 11 is a block diagram of an encoding apparatus according to an embodiment of the present application;

fig. 12 is a schematic structural view of an encoding apparatus of an embodiment of the present application;

FIG. 13 is a flow chart of a decoding method according to an embodiment of the present application;

FIG. 14 is a geometric information reconstruction block diagram;

FIG. 15 is a schematic diagram of a decoding framework for a video-based three-dimensional grid;

FIG. 16 is a block diagram of a decoding apparatus according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of a communication device according to an embodiment of the present application.

Detailed Description

The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the application, fall within the scope of protection of the application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the "first" and "second" distinguishing between objects generally are not limited in number to the extent that the first object may, for example, be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/" generally means a relationship in which the associated object is an "or" before and after.

It should be noted that the techniques described in the embodiments of the present application are not limited to long term evolution (Long Term Evolution, LTE)/LTE evolution (LTE-Advanced, LTE-a) systems, but may also be used in other wireless communication systems, such as code division multiple access (Code Division Multiple Access, CDMA), time division multiple access (Time Division Multiple Access, TDMA), frequency division multiple access (Frequency Division Multiple Access, FDMA), orthogonal frequency division multiple access (Orthogonal Frequency Division Multiple Access, OFDMA), single carrier frequency division multiple access (Single-carrier Frequency Division Multiple Access, SC-FDMA), and other systems. The terms "system" and "network" in embodiments of the application are often used interchangeably, and the techniques described may be used for both the above-mentioned systems and radio technologies, as well as other systems and radio technologies. The following description describes a New air interface (NR) system for purposes of example and uses NR terminology in much of the description that follows, but these techniques are also applicable to applications other than NR system applications, such as generation 6 (6) ^th Generation, 6G) communication system.

The encoding and decoding methods, devices and equipment provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings by some embodiments and application scenarios thereof.

As shown in fig. 1, an embodiment of the present application provides an encoding method, including:

step 101, the encoding end decodes and reconstructs the geometric information of the encoded target three-dimensional grid to obtain reconstructed first geometric information;

it should be noted that, in the present application, the three-dimensional mesh of the target may be understood as a three-dimensional mesh corresponding to any video frame, and the geometric information of the three-dimensional mesh of the target may be understood as coordinates of vertices in the three-dimensional mesh, where the coordinates generally refer to three-dimensional coordinates.

102, the coding end performs repeated vertex screening on the first geometric information to obtain repeated vertex information;

it should be noted that, by filtering the repeated vertices, information of the repeated vertices may be obtained, and at the same time, second geometric information may also be obtained, where the second geometric information is mesh geometric information excluding the repeated vertices, where the repeated vertices are vertices other than the first vertex among the multiple vertices with the same position coordinates, and the first vertex is one vertex among the multiple vertices with the same position coordinates.

Step 103, the coding end reorders the first connection relation of the target three-dimensional grid according to the repeated vertex information, and acquires a reordered second connection relation;

it should be noted that, in the embodiment of the present application, the connection relationship is used to describe a connection relationship between elements such as vertices and patches in the three-dimensional mesh, and may also be referred to as a connectivity relationship.

And 104, the encoding end encodes the second connection relation.

It should be noted that, in the embodiment of the present application, the repeated vertices are removed from the reconstructed geometric information, then the connection relationships are reordered, and the reordered connection relationships are encoded, so that the problem of discontinuous three-dimensional grids caused by the repeated vertices can be avoided, and the encoding efficiency of the connection relationships of the three-dimensional grids is improved.

It should be noted that, in order to obtain the reconstructed geometric information, the geometric information needs to be encoded first, and in an embodiment of the present application, a specific manner of encoding the geometric information of the three-dimensional grid may include the following processes:

step S21, before geometric information encoding, simplifying the target three-dimensional grid to obtain a simplified grid;

It should be noted that this process is an optional step.

Specifically, the encoding end performs simplification processing on the target three-dimensional grid based on quantization parameters when simplifying the target three-dimensional grid to obtain a simplified grid.

It should be noted that, the quantization parameters in the present application mainly include quantization parameters in three components of X-direction, Y-direction and Z-direction.

Optionally, based on the quantization parameter, simplifying the target three-dimensional grid, and obtaining the specific implementation manner of the simplified grid is as follows:

and when the encoding end performs vertex merging in the target three-dimensional grid, adjusting the position coordinate values of part or all of the vertexes after the vertex merging in the target three-dimensional grid to be a multiple of quantization parameters to obtain a simplified grid.

It should be noted that, when the position of the point after merging is set, the quantization parameter is considered, and the quantization parameter is set as a multiple of the quantization parameter, and the multiple can be any value meeting the requirement, so that the original position can be restored without additional information during inverse quantization, and the data amount consumed by high-precision geometric information can be reduced.

The following describes a specific implementation of the simplified process.

For the input original grid, first, a grid simplifying operation is performed. Grid simplification focuses on the simplified operation and the corresponding error metric. The grid reduction operation herein may be edge-based reduction. As shown in fig. 2, the reduction of the number of patches and the number of vertices can be achieved by merging two vertices of one edge. In addition, the grid can be simplified by a grid simplification method based on points and the like.

The simplified error metric needs to be defined during grid simplification. For example, the sum of equation coefficients of all adjacent faces of a vertex can be selected as the error metric of the vertex, and the error metric of the corresponding edge is the sum of error metrics of two vertices on the edge. After determining the manner of simplifying the operation and the error metric, the grid may begin to be simplified. For example, the grid may be divided into one or more partial grids, and the vertex errors of the initial grid in the grid may be calculated to obtain the errors for each edge. All edges within the sheet are then arranged according to some rule, e.g. from small to large, by error. Each simplification can combine edges according to a certain rule, such as selecting the edge with the smallest error for combining, calculating the position of the combined vertex, updating the errors of all edges related to the combined vertex, and updating the sequence of the edge arrangement. The faces of the mesh are reduced to some desired number by iteration.

The specific process comprises the following steps:

1. calculation of vertex error

Vertex error may be defined as the sum of coefficients of equations for all adjacent faces of the vertex. For example, each adjacent surface defines a plane, which can be expressed by equation one:

Equation one: d (D) ² ＝(n ^T v+d) ² ＝v ^T (nn ^T )v+2dn ^T v+d ² ；

Wherein D is the distance from any vertex to the plane, n is the unit normal vector of the plane, v is the position vector of the vertex, and D is a constant. Expressed in the form of a quadric as equation two: q= (a, b, c) = (nn) ^R ,dn,d ² )；

Wherein Q is the vertex error, A, b, c are coefficients representing the corresponding symbols in equation one.

From equation two, equation three is then derived: q (v) =v ^T Av+2b ^T v+c；

The vertex error is the sum of equation coefficients of all adjacent surfaces of the vertex, and then the formula four can be: q (Q) ₁ (v)+Q ₂ (v)＝(Q ₁ +Q ₂ )(v)＝(A ₁ +A ₂ ,b ₁ +b ₂ ,c ₁ +c ₂ ) (v). Wherein Q (v) is the vertex error, v is the corresponding vertex, Q ₁ (v) Equation for v adjacent plane 1, Q ₂ (v) Equation for v adjacent plane 2, A ₁ ，A ₂ ，b ₁ ，b ₂ ，c ₁ ，c ₂ For each corresponding coefficient. Of course, if there are multiple adjacent facets, then a corresponding plane error equation may be continually added to equation four.

2. Merging vertices

One of the main steps in the process of merging vertices is to determine the location of the merged vertices. According to the error formula three, the vertex position that can make the error as small as possible can be selected. The example pair can be obtained by taking the partial derivative of the formula three, formula four:

formula five:

as can be seen from the above equation, the point at which the error is minimized can be obtained only when the matrix a is reversible. Thus, there are a number of ways to obtain the position of the vertices after merging. If the quality of grid simplification is considered, under the condition that the matrix A is reversible, selecting the vertex position with the minimum error; in the case where matrix a is irreversible, one of the points on the edge that include two end points can be selected to minimize the error. If the complexity of mesh simplification is considered, the midpoint of the edge or one of the two end points can be directly selected as the position of the vertex after merging. If the efficiency of mesh reduced quantization is considered, the position of the combined vertices also needs to be adjusted. Because the high-precision information needs to be independently encoded after quantization, the positions of the vertexes after a part of merging are adjusted to be multiples of corresponding quantization parameters, so that the original positions can be restored without additional information during inverse quantization, and the data quantity consumed by the high-precision geometric information can be reduced.

After determining how to select the merged vertex position, the process of merging vertices may begin. For example, the errors of all edges in the initial grid may be calculated first and arranged in order of some specification, e.g., from small to large. For each iteration, selecting the edge with the error meeting a certain rule, such as the edge with the minimum error. Two end points of the edge are removed from the mesh vertices and the merged vertex is added to the set of mesh vertices. And taking all or part of adjacent vertexes of the two vertexes before merging as adjacent vertexes of the merged vertexes, and updating error metrics of all points connected with the merged vertexes so as to obtain errors of newly generated edges. The order of the edges is then updated from the global of the tile. The above process is looped until the number of faces required to meet the lossy coding is reached.

3. Updating connection relationships

After merging the vertices, since a part of the vertices are deleted in the vertex set and a number of new vertices are added, the connection relationship between the vertices needs to be updated. For example, two vertices before merging corresponding to the merged vertex may be determined during merging of vertices. The aim of updating the connection relation can be achieved by replacing the indexes of all the two vertices before merging, which occur in the faces, with the indexes of the vertices after merging, and then deleting the faces with repeated indexes.

The above is the main process of grid simplification. Meanwhile, the three-dimensional grid may also carry attribute information, and the attribute information may also need to be simplified. For grids with attribute information, such as texture coordinates, colors, normal vectors, etc., vertex coordinates can be extended to higher dimensions to calculate vertex errors with attribute information. Taking texture coordinates as an example, let vertex coordinates be (x, y, z) and texture coordinates be (u, v), the extended vertex be (x, y, z, u, v). Let the extended triangle t= (p, q, r), to determine the error metric in high-dimensional space, two orthonormal vectors are first calculated, namely:

formula six:

formula seven:

wherein e ₁ ，e ₂ Two vectors on the plane of T, q is e ₂ For purposes of this document ". Cndot." represents the dot product of a vector, which defines a coordinate axis on the high-dimensional plane, with p as the origin. Consider an arbitrary point v, another u=p-v, represented by equation eight: II u II ² ＝(u·e ₁ ) ² +…+(μ·e _n ) ² ；

Namely formula nine: (u.e) ₃ ) ² +…+(u·e _n ) ² ＝‖μ‖ ² -(μ·e ₁ ) ² -(u·e ₂ ) ² 。

Due to e ₁ ，e ₂ For two vectors on the plane where T is located, the term on the left of formula nine is the square of the distance from the vertex to the plane where T is located, namely formula ten: d (D) ² ＝‖μ‖ ² -(μ·e ₁ ) ² -(u·e ₂ ) ² ；

After being unfolded and combined, the method can obtain an equation similar to a third equation, wherein:

Formula eleven:

formula twelve: b= (p.e ₁ )e ₁ +(p·e ₂ )e ₂ -p；

Formula thirteen: c=p.e- (p.e) ₁ ) ² -(p·e ₂ ) ² 。

After the error measurement is obtained, the subsequent steps which are the same as the previous three-dimensional information can be performed, so that the grid with the attribute information is simplified.

Generally, the edge portion of an image attracts more attention, thereby affecting the quality evaluation of the image. As is the case with three-dimensional grids, one tends to notice the boundary portions more easily. Therefore, whether to maintain a boundary is also a factor in mesh simplification that affects quality. The boundaries of the mesh are typically geometric boundaries as well as texture boundaries. When an edge belongs to only one face, the edge is a geometric boundary. When the same vertex has two or more texture coordinates, the vertex is the boundary of the texture coordinates. None of the above boundaries should be merged when the grid is simplified. Therefore, in each simplification, whether the vertex on the edge is a boundary point can be judged first, if so, the vertex is skipped, and the next iteration is directly carried out.

S22, quantifying the geometric information of the simplified grid to obtain first information;

it should be noted that the geometric information of the simplified mesh may be understood as coordinates of vertices in the three-dimensional mesh, and the coordinates are generally referred to as three-dimensional coordinates.

Specifically, the first information includes at least one of:

a11, first precision geometric information;

it should be noted that, the first precision geometric information may be understood as low precision geometric information, that is, low precision geometric information refers to geometric information of the quantized target three-dimensional grid, that is, three-dimensional coordinate information of each vertex included in the quantized target three-dimensional grid.

A12, second precision geometric information;

it should be noted that, the second precise geometric information may be understood as high-precise geometric information, and the high-precise geometric information may be regarded as missing geometric information, i.e., missing three-dimensional coordinate information, in the quantization process.

A13, supplementing information of points;

the information of the supplemental points refers to information of points requiring additional processing generated in the quantization process, that is, the supplemental points are points requiring additional processing generated in the quantization process, for example, overlapping repeat points of the coordinate positions, and the like, and by processing the repeat points, vertices overlapping the coordinate positions in the quantization can be restored to the original positions after the inverse quantization.

Optionally, the information of the supplemental point includes at least one of:

a131, the index of the vertex in the first precision geometric information corresponding to the supplementary point;

It should be noted that, by identifying the index, it is possible to know which points in the quantized grid are identified by a plurality of points in the three-dimensional grid before quantization, that is, the plurality of points in the three-dimensional grid before quantization are overlapped together after quantization, and the low-precision geometric information of the complementary points can be determined by the index of the vertex.

A132, supplementing third-precision geometric information of the points;

it should be noted that the third precision geometric information may be understood as low precision geometric information of the supplemental point, that is, three-dimensional coordinate information after the supplemental point is quantized.

A133, supplementing fourth-precision geometric information of the points;

it should be noted that the fourth precise geometric information may be understood as high-precise geometric information of the supplemental point, that is, three-dimensional coordinate information of the supplemental point that is lost in the quantization process.

It should be noted that, in specific use, it is possible to determine which of the hidden points after quantization is obtained through a131 and a133 or through a132 and a 133.

Optionally, in an embodiment of the present application, the specific implementation manner of step S22 is:

and the encoding end quantizes each vertex in the simplified grid according to the quantization parameter of each component to acquire first precision geometric information.

It should be noted that the quantization parameter of each component can be flexibly set according to the use requirement.

In general, for quantization with low precision requirement, only low-precision geometric information can be reserved after quantization; for quantization with higher precision requirement, not only low-precision geometry information but also high-precision geometry information needs to be recorded during quantization, so that accurate grid recovery can be realized during decoding, that is, the specific implementation manner of step S22 should further include:

the encoding end obtains second precision geometric information according to the first precision geometric information and the quantization parameter of each component.

For example, assuming that the three-dimensional coordinates of a vertex are (x, y, z), the quantization parameter is (QP _x ,QP _y ,QP _z ) Low-precision geometric information (x _l ,y _l ,z _l ) And high precision geometric information (x _h ,y _h ,z _h ) The calculation process of (1) is shown as formula fourteen to formula nineteenth:

formula fourteen: x is x _l ＝f ₁ (x,QP _x )；

Formula fifteen: y is _l ＝f ₁ (y,QP _y )；

Formula sixteen: z _l ＝f ₁ (z,QP _z )；

Seventeenth formula: x is x _h ＝f ₂ (x,x _l ,QP _x )；

Equation eighteen: y is _h ＝f ₂ (y,y _l ,QP _y )；

Nineteenth formula: z _h ＝f ₂ (z,z _l ,QP _z )；

Wherein f in formulas fourteen to sixteen ₁ The function is a quantization function, the input of the quantization function is the coordinate of a certain dimension and the quantization parameter of the dimension, and the output is the quantized coordinate value; f in formulas seventeen to nineteen ₂ The function input is the original coordinate value, the quantized coordinate value and the quantized parameter of the dimension, and the function input is the high-precision coordinate value.

f ₁ There are a number of ways of calculating the function, one more common way of calculating is shown in equations twenty-two, using the original coordinates of each dimension divided by the quantization parameter for that dimension. Where/is a division operator, the result of the division operation may be rounded in different ways, such as rounding, rounding down, rounding up, etc. f (f) ₂ There are also various ways of calculating the function, and the implementations corresponding to the seventeen to nineteen equations are shown as twenty-three to twenty-five equations, where x is the multiplication operator.

Formula twenty: x is x _l ＝x/QP _x ；

Formula twenty-one: y is _l ＝y/QP _y ；

Formula twenty-two: z _l ＝z/QP _z ；

Twenty-third formula: x is x _h ＝x-x _l *QP _x ；

Twenty-four of the formulas: y is _h ＝y-y _l *QP _y ；

Twenty-five of the formula: z _h ＝z-z _l *QP _z ；

When the quantization parameter is the integer power of 2, f ₁ Function sum f ₂ The function may be implemented using bit operations, such as the formulas twenty-six through thirty-one:

formula twenty-six: x is x _l ＝x＞＞log ₂ QP _x ；

Seventeenth formula: y is _l ＝y＞＞log ₂ QP _y ；

Formula twenty-eight: z _l ＝z＞＞log ₂ QP _z ；

The formula twenty-nine: x is x _h ＝x&(QP _x -1)；

The formula thirty: y is _h ＝y&(QP _y -1)；

Formula thirty one: z _h ＝z&(QP _z -1)；

Notably, no matter f ₁ Function sum f ₂ Which calculation mode is adopted by the function, and quantization parameter QP _x 、QP _y And QP (QP) _z Can be flexibly set. First, the quantization parameters of different components are not necessarily equal, and the QP can be established by utilizing the correlation of the quantization parameters of different components _x 、QP _y And QP (QP) _z The relation between the two sets of quantization parameters for different components; and secondly, the quantization parameters of different space regions are not necessarily equal, and the quantization parameters can be adaptively set according to the sparseness degree of the vertex distribution of the local region.

The high-precision geometric information includes detailed information of the outline of the three-dimensional mesh. To further improve compression efficiency, high precision geometric information (x _h ,y _h ,z _h ) And (5) further processing. In a three-dimensional mesh model, the degree of importance of high-precision geometric information of vertices in different regions is different. For areas with sparse vertex distribution, the distortion of high-precision geometric information does not have great influence on the visual effect of the three-dimensional grid. In order to improve the compression efficiency, the high-precision geometric information can be further quantized, or only partial high-precision geometric information of points can be reserved.

Optionally, in the process of quantization, there may be a plurality of points that are quantized and completely overlapped to the same position, that is, in this case, the specific implementation of step S22 should further include:

And the coding end determines the information of the supplementary points according to the geometric information of the simplified grid and the first precision geometric information.

That is, after low-precision geometric information of all vertices is obtained, points at which the low-precision geometric information is repeated are used as supplemental points, and encoding is performed individually. The geometric information of the supplementary points can be divided into two parts, namely low-precision geometric information and high-precision geometric information, and all supplementary points or only part of supplementary points can be selected to be reserved according to the requirement of application on compression distortion. Further quantization may also be performed on the high-precision geometric information of the supplemental points, or only high-precision geometric information of a portion of the points may be retained.

It should be noted that after the geometric information of the target three-dimensional grid is quantized to obtain the first information, the first information needs to be encoded to obtain a final code stream, and optionally, the specific implementation process for encoding the first information in the embodiment of the present application includes:

step S221, the encoding end processes the first information to obtain second information, wherein the second information comprises at least one of a occupation map and a geometric figure;

in step S222, the encoding end encodes the second information.

In addition, since the types of information included in the first information are different, when the first information is processed, the different types of information are separately processed, and the implementation procedure of step S221 is described below from the standpoint of different information.

1. The first information includes first precision geometric information

Optionally, in this case, the specific implementation procedure of step S221 includes:

step S2211, the coding end performs three-dimensional slice division on the first precision geometric information;

in this case, the low-precision geometric information is divided into a plurality of three-dimensional slices (Patch); the specific implementation mode of the step is as follows: the coding end determines a projection plane of each vertex contained in the first precision geometric information; the encoding end performs slice division on the vertexes contained in the first precision geometric information according to the projection plane; and the encoding end clusters the vertexes contained in the first precision geometric information to obtain each divided piece. That is, the process for Patch partitioning mainly includes: firstly, estimating the normal vector of each vertex, and selecting a candidate projection plane with the minimum included angle between the plane normal vector and the vertex normal vector as the projection plane of the vertex; then, carrying out initial division on vertexes according to the projection planes, and forming patch by the vertexes which are identical and communicated with the projection planes; and finally, optimizing the clustering result by using a fine division algorithm to obtain a final three-dimensional patch (3D patch).

A detailed description of a specific implementation of the process of obtaining the three-dimensional slice from the first precision geometric information follows.

The normal vector for each point is first estimated. The tangential plane and its corresponding normal are defined at a predefined search distance from the nearest neighbor vertex m of each point. K-D tree is used to separate data and is at point p _i Find adjacent points nearby, the center of gravity of the setFor defining a normal. The center of gravity c is calculated as follows:

the formula thirty-two:

the vertex normal vector is estimated by using a characteristic decomposition method, and a calculation process formula thirty-three is shown as follows:

the formula thirty-three:

in the initial stage of the division into the segments,a projection plane for each vertex is initially selected. Let the estimated value of the vertex normal vector beThe normal vector of the candidate projection plane is +.>Selecting a plane closest to the normal vector direction of the vertex as a projection plane of the vertex, wherein the calculation process of plane selection is shown as a thirty-fourth formula:

the formula thirty-four:

the fine division process may employ a grid-based algorithm to reduce the time complexity of the algorithm, and the grid-based fine division algorithm flow is shown in fig. 3, and specifically includes:

firstly, setting the circulation number (numlter) to 0, judging whether the circulation number is smaller than the maximum circulation number (the maximum circulation number can be set according to the use requirement), and if so, executing the following processes:

In step S301, the (x, y, z) geometric coordinate space is divided into voxels.

The geometric coordinate space herein refers to a geometric coordinate space formed by the quantized first precision geometric information. For a 10 bit Mesh with a voxel size of 8, for example, the number of voxels per coordinate would be 1024/8=128, the total number of voxels in this coordinate space will be 128 x 128.

In step S302, a filling voxel is searched, where the filling voxel is a voxel containing at least one point in the grid.

In step S303, a smoothing score of each filling voxel on each projection plane is calculated, denoted as voxscore smoothing, and the voxel smoothing score of a voxel on a projection plane is the number of points that are aggregated on the projection plane by the initial segmentation process.

Step S304, using KD-Tree partitioning to find neighboring filler voxels, denoted as nnfiledwoxels, i.e. the nearest filler voxel of each filler voxel (within the search radius and/or limited to the maximum number of neighboring voxels).

Step S305, calculating a smoothing score (score smoothing) of each filling voxel by using voxel smoothing scores of neighboring filling voxels in each projection plane, wherein the calculation process is as shown in the formula thirty-five:

Thirty-five of the formula:

where p is the index of the projection plane and v is the index of the neighbor filling voxels. Score smooth is the same for all points in a voxel.

Step S306, using the normal vector of the vertex and the normal vector algorithm vector score of the candidate projection plane, which is denoted as score normal, the calculation process is shown as a thirty-six formula:

the formula thirty-six: score normal [ i ] [ p ] =normal [ i ] & orientation [ p ];

where p is the index of the projection plane and i is the index of the vertex.

Step S307, calculating the final score of each voxel on each projection plane by using the score smooth and score normal, wherein the calculation process is shown as a thirty-seventh formula:

seventeenth formula:

where i is the vertex index, p is the index of the projection plane, and v is the voxel index where vertex i is located.

Step S308, clustering the vertexes by using the scores in step 307 to obtain finely divided patches.

The above process is iterated multiple times until a more accurate patch is obtained.

Step S2212, the coding end carries out two-dimensional projection on the divided three-dimensional slices to obtain two-dimensional slices;

it should be noted that this process is to project the 3D patch onto a two-dimensional plane to obtain a two-dimensional patch (2D patch).

Step S2213, the coding end packs the two-dimensional slice to obtain two-dimensional image information;

It should be noted that, this step is implemented by slice packing (Patch packing), where the purpose of Patch packing is to arrange 2d patches on a two-dimensional image, and the basic principle of Patch packing is to arrange patches on a two-dimensional image in a non-overlapping manner or to arrange non-pixel portions of patches on a two-dimensional image in a partially overlapping manner, so that the Patch arrangement is more compact and has time domain consistency and improves coding performance through algorithms such as priority arrangement and time domain consistency arrangement.

Assuming that the resolution of the two-dimensional image is WxH, the minimum block size of the patch arrangement is defined as T, which specifies the minimum distance between different patches placed on this 2D grid.

First, patch is placed on the 2D mesh in an insert-placement manner on a non-overlapping basis. Each patch occupies an area consisting of an integer number of TxT blocks. In addition, at least one TxT block distance is required between adjacent patches. When there is insufficient space to place the next patch, the height of the image will become 2 times the original, and then continue to place the patch.

To make the patch arrangement more compact, the patch may select a variety of different arrangement orientations. For example, eight different alignment directions may be employed, including 0 degrees, 180 degrees, 90 degrees, 270 degrees, and mirror images of the first four directions, as shown in FIG. 4.

To obtain better adaptation to the inter-frame prediction characteristics of a video encoder, a Patch permutation method with temporal consistency is used. In one GOF (Group of frame), all patches of the first frame are arranged in order from large to small. For other frames in the GOF, the order of the patch is adjusted using a time domain consistency algorithm.

It should be noted that, after the two-dimensional image information is obtained, the patch information can be obtained according to the information in the process of obtaining the two-dimensional image information, and then the encoding of the slice information can be performed to obtain the sub-code stream of the slice information.

Here, it should be noted that, in the process of performing two-dimensional image information, information of a patch division, information of a patch projection plane, and information of a patch packaging position need to be recorded, so that the patch information records information of operations of each step in the process of acquiring the two-dimensional image, that is, the patch information includes: information of patch division, information of a patch projection plane and information of a patch packaging position.

Step S2214, the coding end obtains a first-precision occupation map and a first-precision geometric figure according to the two-dimensional image information;

it should be noted that, for the process of obtaining the occupancy map, mainly: and setting the position of the vertex in the two-dimensional image as 1 and the rest positions as 0 by using the patch arrangement information obtained by patch packing to obtain a occupation map. For the process of acquiring geometry, mainly: in the process of obtaining the 2D patch through projection, the distance from each vertex to a projection plane is saved, the distance is called depth, and the low-precision geometric figure compression part is used for arranging the depth value of each vertex in the 2D patch to the position of the vertex in the occupied figure to obtain the low-precision geometric figure.

2. The first information includes second precision geometric information

step S2215, the encoding end obtains the arrangement sequence of the vertexes contained in the first precision geometric information;

in step S2216, the encoding end arranges the second precision geometric information corresponding to the vertex included in the first precision geometric information in the two-dimensional image, and generates a second precision geometric figure.

The high-precision geometric information adopts an arrangement mode of original sheets (raw patch), and the high-precision geometric information corresponding to the vertexes in the low-precision geometric figure is arranged in a two-dimensional image to obtain the raw patch, so that the high-precision geometric figure is generated. The method mainly comprises three steps, as shown in fig. 5, including:

step 501, obtaining the vertex arrangement sequence, scanning the low-precision geometric figure row by row from left to right, and taking the scanning sequence of each vertex as the arrangement sequence of the vertices in the row patch.

Step 502, generate a raw patch.

Note that, the raw patch is a rectangular patch formed by arranging three-dimensional coordinates of vertices line by line in a manner as shown in fig. 6. And (3) sequentially arranging the high-precision geometric information of the vertexes according to the vertex arrangement sequence obtained in the first step to obtain high-precision geometric information raw patch.

In step 503, the high-precision geometric information is placed in a two-dimensional image, and a high-precision geometric figure is generated.

When the geometric figure sub-code stream is obtained by encoding, the encoding end encodes the geometric figure with the first precision and the geometric figure with the second precision to obtain the geometric figure sub-code stream.

3. The first information includes information of the supplemental point

step S2217, the coding end arranges the third precision geometric information of the supplementary points into a first original sheet;

step S2218, the coding end arranges the fourth precision geometric information of the supplementary points into a second original slice according to the arrangement sequence same as the first original slice;

in step S2219, the encoding end compresses the first original slice and the second original slice to obtain a geometric figure of the supplemental point.

In the embodiment of the present application, the low-precision part and the high-precision part, which are divided into the geometric information of the supplemental points, are respectively encoded. Firstly, arranging low-precision geometric information of supplementary points into a low-precision raw patch of the supplementary points according to any sequence; then, arranging the high-precision geometric information into the high-precision raw patch of the supplementary point according to the same sequence as the low-precision raw patch of the supplementary point; finally, the low-precision raw patch and the high-precision raw patch of the supplementary points are compressed, and various compression methods can be adopted. One method is to encode the value in the raw patch in a running mode, an entropy encoding mode and the like, and the other method is to add the low-precision raw patch of the supplementary point into a blank area in the low-precision geometric figure, and add the high-precision raw patch of the supplementary point into the blank area in the high-precision geometric figure to obtain the geometric figure of the supplementary point.

It should be noted that, after obtaining the encoded geometric information, the geometric information may be reconstructed, the reconstructed first geometric information is obtained, and then the repeated vertex screening is performed on the first geometric information to obtain the repeated vertex information.

The coding end obtains repeated vertexes in the first geometric information and records indexes of the repeated vertexes to form repeated vertex information;

the repeated vertex information comprises indexes of repeated vertexes and texture coordinate indexes of the first vertexes and the repeated vertexes.

It should be noted that, by acquiring the repeated vertices one by one and recording the indexes of the repeated vertices, accurate information of the repeated vertices can be ensured to be acquired.

Optionally, an alternative implementation of step 103 includes:

step 1031, the coding end rearranges the vertex texture coordinates of the target three-dimensional grid according to the indexes of the repeated vertices, and acquires the reordered texture coordinate information;

optionally, the specific implementation manner of this step is: and the coding end sequentially moves the texture coordinates corresponding to the repeated vertexes to the tail part of the list corresponding to the texture coordinates according to the sequence of the repeated vertexes in the first geometric information, so as to form reordered texture coordinate information.

For example, texture coordinates of 10 vertices (vertex 1 to vertex 10) are, in order from front to back: coordinate 1, coordinate 2, coordinate 3, coordinate 4, coordinate 5, coordinate 6, coordinate 7, coordinate 8, coordinate 9, and coordinate 10, and vertex 1 and vertex 6 are repeated, and the texture coordinates of 10 vertices are sequentially from front to back: coordinate 1, coordinate 2, coordinate 3, coordinate 4, coordinate 5, coordinate 7, coordinate 8, coordinate 9, coordinate 10, coordinate 6.

It should be noted that, by using the sequence of the repeated vertices in the first geometric information, the texture coordinate information is reordered, so that it can be ensured that the other parts except the tail parts in the reordered texture coordinate information have no texture coordinate information of the repeated vertices, and the adjustment of the subsequent connection relationship is facilitated.

Step 1032, the encoding end adjusts the first connection relation of the target three-dimensional grid according to the reordered texture coordinate information and the repeated vertex information, and obtains a second connection relation after reordering;

it should be noted that, by reordering the connection relationship, the processed connection relationship does not include the information of the repeated vertexes, so that the problem of discontinuous three-dimensional grids caused by the repeated vertexes can be avoided when the connection relationship is encoded, and the encoding efficiency of the connectivity relationship of the three-dimensional grids is improved.

Since the connection relationship is composed of the texture coordinate index portion and the geometric index portion, it is necessary to adjust the connection relationship when adjusting the connection relationship. Optionally, in an embodiment of the present application, the specific implementation procedure of the step 1032 is:

step 10321, the encoding end modifies the texture coordinate index part of the first connection relation of the target three-dimensional grid according to the reordered texture coordinate information;

step 10322, the encoding end traverses the geometric index portion of the first connection relationship of the target three-dimensional mesh, replaces the index of the repeated vertex of the first connection relationship of the target three-dimensional mesh with the index of the first vertex according to the repeated vertex information, and obtains the second connection relationship after reordering.

For example, the connection relation of 10 vertices (vertex 1 to vertex 10) represents that: (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), wherein the former value in brackets represents the geometric index of the vertex and the latter value represents the texture coordinate index, and because vertex 1 and vertex 6 are repeated, the connection relationship after reordering represents the following steps from front to back: (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (1, 10), (6, 6), (7, 7), (8, 8), (9, 9).

It should be noted that, in the embodiment of the present application, the specific implementation process of encoding the second connection relationship is:

step S31, the encoding end determines vertexes to be ordered in a first space range according to the space included angles of adjacent triangular patches in the three-dimensional grid, wherein the first space range is the space range of a target vertex of a triangle to be encoded in the three-dimensional grid, and the vertex to be ordered comprises the target vertex.

The connection relation coding of the embodiment of the application is a connection relation coding mode driven based on geometric information, wherein the reconstructed geometric information comprises index information of vertexes in a three-dimensional grid.

In the embodiment of the present application, the three-dimensional network may be divided into at least one triangular patch, and each triangular patch includes at least one triangle.

In this step, based on the space included angle between adjacent triangular patches in the three-dimensional grid, part of the vertices in the first space range may be screened out, and the remaining part of the vertices may be used as vertices to be ordered.

Optionally, the first spatial range includes:

a spatial extent between the first sphere and the second sphere;

the centers of the first sphere and the second sphere are the same, the radius of the first sphere and the radius of the second sphere are different, and the center of the sphere is a target position in the first side of the triangle to be encoded, for example, the center of the sphere is the midpoint of the first side of the triangle to be encoded.

And S32, the encoding end orders the vertexes to be ordered to obtain ordering information of the target vertexes.

In this step, the vertices to be sorted are sorted according to a preset sorting criterion, for example, the sorting criterion may be sorting according to a distance between the vertices and a midpoint of a first edge of the triangle to be encoded, or the sorting criterion may also be sorting according to a radius of a triangle circumscribing circle formed by the vertices and the first edge. Of course, the ranking criteria may be other criteria, and are not specifically limited herein.

Here, by further deleting the vertices within the first spatial range by the spatial angles of the adjacent triangular patches, the number of vertices to be ordered is reduced, i.e. the bit information used for ordering information of the target vertices can be reduced,

and step S33, the coding end obtains the coding information of the triangle to be coded according to the coding information corresponding to the ordering information of the target vertex.

Optionally, the encoding end encodes the ordering information of the target vertex under the condition that the target condition is met, so as to obtain the encoding information, and encodes the index of the target vertex under the condition that the target condition is not met, so as to obtain the encoding information. For example, the target condition is that the number of vertices to be ordered in the first spatial range is smaller than a preset threshold value, and/or the ordering sequence number of the target vertices is smaller than a preset value. Because the number of the vertexes to be ordered in the first space range is smaller or the ordering sequence number of the target vertexes is smaller, the encoding information corresponding to the ordering information occupies smaller bit number, and at the moment, the ordering information of the target vertexes is encoded, so that the encoding bit number can be effectively reduced. And for the case that the number of the vertexes to be ordered is larger or the ordering sequence number of the destination vertexes is larger, compared with the case that the ordering information is encoded, the encoding bit number can be effectively reduced when the index of the target vertexes is encoded.

In the embodiment of the application, according to the space included angle of the adjacent triangular patches in the three-dimensional grid, partial vertexes are eliminated in the first space range, and the vertexes to be ordered are determined based on the eliminated vertexes, namely, the number of the vertexes to be ordered is reduced, so that when the ordering information of the target vertexes is encoded, the bit number occupied by the encoding information can be further reduced, and the encoding efficiency is further effectively improved.

Optionally, before the encoding end determines the first spatial range, the method further includes:

the encoding end selects a first edge from edge sets corresponding to the three-dimensional grid, wherein the edge sets are sets of at least one edge of the encoded triangle in the three-dimensional grid;

the encoding end determines a triangle to be encoded according to the first edge and the vertexes corresponding to the first edge, wherein the target vertexes of the triangle to be encoded are vertexes except for two vertexes connected with the first edge in the vertexes corresponding to the first edge, and the target vertexes can be described as pairs of vertexes of the first edge.

Optionally, the determining, by the encoding end, the vertex to be ordered in the first spatial range according to the spatial included angle of the adjacent triangular patches in the three-dimensional grid includes:

And the coding end determines vertexes to be ordered in a first space range according to the space included angles of adjacent triangular patches in the three-dimensional grid under the condition that the triangle to be coded is a triangle except a preset class triangle.

Optionally, the preset category triangle includes at least one of the following:

triangles with angles smaller than a preset angle with the coded triangles;

triangle with two coincident vertexes or three collinear vertexes. In particular, it means that two vertexes in a triangle coincide or three vertexes are collinear.

Optionally, the method of the embodiment of the present application further includes:

and the coding end obtains the coding information of the triangle to be coded according to the coding information corresponding to the target vertex information of the triangle to be coded under the condition that the triangle to be coded is the triangle of the preset class.

For example, if the triangle to be encoded is the preset class triangle, directly encoding the index of the target vertex of the triangle to be encoded, and obtaining the encoding information of the triangle to be encoded according to the encoding information corresponding to the index of the target vertex.

In the embodiment of the present application, when the index of the vertex is encoded, binary representation may be directly used or an encoding algorithm such as huffman may be used for encoding, and the encoding method is not particularly limited herein.

Optionally, after the obtaining the coding information of the triangle to be coded, the method further includes:

the encoding end updates the edge set according to a first preset rule;

the encoding end redetermines the triangle to be encoded according to the updated edge set until the triangles in the three-dimensional grid all obtain encoding information;

wherein, the first preset rule includes: adding two edges of the triangle to be encoded except the first edge to the edge set, and removing the first edge from the edge set.

the encoding end excludes all vertexes of the first target triangle from vertexes in the first space range to obtain residual vertexes;

the encoding end determines vertexes to be ordered in the first space range according to the residual vertexes;

the first target triangle is a triangle with an included angle smaller than the included angle threshold value with the adjacent coded triangle, and one side of the first target triangle is the same as the first side of the triangle to be coded.

Optionally, the encoding information of the triangle to be encoded further includes: and the included angle threshold value is encoded information.

Here, the included angle threshold is encoded, so that the decoding end can obtain the included angle threshold according to the encoded information, and determine the vertices to be ordered in the first spatial range based on the included angle threshold.

Of course, a fixed included angle threshold value can be pre-agreed, the coding end and the decoding end determine the vertices to be ordered in the first space range based on the pre-agreed included angle threshold value, and the coding end does not need to code the included angle threshold value.

Optionally, the method of the embodiment of the present application further includes: and encoding target vertex information of the triangle to be encoded in a second spatial range to obtain encoding information of the triangle to be encoded, wherein the second spatial range is a range except the first spatial range in the three-dimensional grid.

Optionally, the coding information of the triangle to be coded further includes coding information of the first spatial range.

For example, the radii of the first sphere and the second sphere are encoded, and the first spatial range may be flexibly set in this implementation manner.

Of course, the encoding end and the decoding end may also predetermine the size of the first spatial range, and in this manner, the encoding end does not need to encode the first spatial range.

In a specific embodiment of the application, the input three-dimensional mesh is divided into one or more slices at the encoding end, and an initial triangle is selected in each slice. The vertex index of the initial triangle is encoded and the edges of the initial triangle are placed into a set of edges (i.e., a set of edges). Selecting one side in the side set and determining the opposite vertex of the side set, wherein the triangle formed by the side and the opposite vertex is the triangle to be encoded. For example, the edge chosen for each iteration may be denoted τ, its pair of vertices denoted v, and the triangle to be encoded adjacent to that edge is encoded. As shown in fig. 7 and 8, the process of encoding the connection relationship may specifically include:

(1) Under the condition that the preset condition is met, the vertex index of the triangle can be directly encoded, and other modes of encoding can be performed on the triangle. The preset condition may be that the triangle to be encoded belongs to a plurality of special triangles, for example, belongs to a degradation surface (two points overlap or three points are collinear) or an included angle between the triangle and the encoded triangle is smaller than a certain angle, or the preset condition is that the number of vertices in the first space range is larger than a preset number, or the preset condition is that the target vertex is located outside the second space range, for example, is located in the second space range, or the sequence number of the target vertex in the vertices in the first space range in the preset condition is larger than or equal to a preset value, and the preset condition can be flexibly set according to requirements. Adding two edges of the triangle to be encoded except for the edge tau to the edge set, and removing the edge tau from the set. Then another edge (the edge except the edge tau) of the triangle to be coded is taken out from the edge set according to a certain criterion, and the triangle adjacent to the edge is continuously coded, for example, the next edge tau can be selected by adopting the access sequence.

(2) If the preset condition is not satisfied, determining the spatial range (i.e., the first spatial range) in which the vertex v is located, and encoding the spatial range. And traversing all vertexes in the space range in the slice, screening out all vertexes of a new triangle which forms an included angle smaller than a certain angle with the coded adjacent triangle with the side tau, and coding the angle value.

The spatial extent may be determined using geometric characteristics of adjacent triangular patches, spatial angles, or other criteria. For example, the spatial range may be defined by a center of sphere at the midpoint of the side τ, with a minimum radius R _min And a maximum radius R _max Combination { R _min ,R _max The part between two concentric spheres, encoded with { R } _min ,R _max Sets.

Alternatively, the above-mentioned angle values may also be encoded.

(3) Traversing all vertexes in the space range, and sorting according to a certain sorting criterion. For example, the ranking criterion may be the distance of the vertex v to the midpoint of the edge τ; or the radius of the circumcircle of the triangle formed with the side τ. The code sequences the vertex v in the order.

(4) Adding two edges of the newly encoded triangle except for the edge tau to the edge set, and removing the edge tau from the edge set. And then the other edge of the newly encoded triangle is taken out from the set according to a certain criterion, and the triangle adjacent to the edge is continuously encoded.

The encoding process is iterated for each tile of the three-dimensional mesh until the triangles in each tile are encoded. If the edge set is empty but there are not yet encoded triangles, then an initial triangle is selected from the remaining unencoded triangles, and the encoding process is cycled.

It should be further noted that, in order to ensure consistency of the connection relationships, in an embodiment of the present application, the encoding end adjusts a portion where the connection relationship of the target three-dimensional grid changes according to the reconstructed first geometric information, and performs encoding of the second connection relationship after reordering based on the adjusted connection relationship.

It should be noted that, the embodiment of the present application further includes encoding the repeated vertices, and the specific implementation manner is as follows:

the encoding end queries whether repeated vertexes exist for three vertexes of each triangular surface patch according to repeated vertex information in the process of encoding the second connection relation;

the coding end marks the repeated vertexes under the condition that the repeated vertexes exist;

and after the encoding end completes the encoding of the second connection relation, encoding according to the identification of the repeated vertexes to obtain a repeated vertex identification code stream.

The accuracy of the repeated vertex coding can be ensured by marking the repeated vertices in the connection relationship based on the repeated vertex information and then coding by using the marks.

That is, in the process of encoding the connection relation of the three-dimensional mesh, for three vertices of each triangular patch, whether there are repeated vertices is inquired according to the repeated vertex information; if the repeated vertex exists, the repeated vertex is identified, and after the encoding of the connection relation is completed, the repeated vertex identification is encoded into a repeated point identification code stream by using an entropy encoder.

The application also needs to encode texture coordinates (UV coordinates), wherein the UV coordinates are information describing three-dimensional grid vertex textures, the three-dimensional grid firstly carries out two-dimensional projection on the surface textures to form a two-dimensional texture map, the UV coordinates represent positions in the two-dimensional texture map where the three-dimensional vertex textures are located and correspond to the geometric information one by one, and optionally, the implementation process of the UV coordinate encoding in an embodiment of the application is as follows:

the coding end recovers the repeated vertexes in the second connection relation with the second geometric information according to the sequence of the repeated vertexes in the reordered texture coordinate information;

The coding end traverses the second connection relation of the recovered repeated vertexes and records the traversing sequence;

and the encoding end encodes texture coordinates of the vertexes according to the traversing sequence.

It should be noted that, since the connection relationship is processed to remove the repeated vertices in the above process, and the texture coordinate encoding is performed based on the connection relationship and the geometric information including the repeated vertices, it is necessary to recover the repeated vertices in the second connection relationship and the geometric information first to ensure the accuracy of the texture coordinate encoding.

In the embodiment of the present application, the texture coordinates of the vertices are encoded by using the third geometry information (the geometry information corresponding to the second geometry information after the repeated vertex recovery) obtained by recovering the repeated vertices in the second geometry information and the second connection relationship by using the reordered texture coordinate information (i.e., the connection relationship corresponding to the second connection relationship after the repeated vertex recovery).

The specific implementation process comprises the following steps:

step S41, the encoding end determines a target triangle according to a third connection relation, wherein the target triangle is composed of a first edge and a vertex to be encoded;

Optionally, in an embodiment of the present application, an optional implementation manner of this step is:

the encoding end selects a first edge from an edge set, wherein the edge set is a set formed by edges of triangles constructed by a third connection relation; and the coding end determines a target triangle according to the first edge.

Optionally, before the encoding end selects the first edge in the edge set, the method further includes:

the coding end selects an initial triangle according to a third connection relation; the encoding end encodes texture coordinates of three vertexes of the initial triangle, and adds three sides of the initial triangle into a side set.

Typically, the initial triangle is the first triangle in the connection. For the initial triangle, the vertex prediction is not performed, but texture coordinates are directly encoded, after the texture coordinates of each vertex of the initial triangle are encoded (it should be noted that, the texture coordinates refer to the original texture coordinates directly obtained through the target three-dimensional grid), each side of the initial triangle is added to the side set to form an initial side set, and then the following vertices are predicted based on the initial side set.

Step S42, the encoding end predicts the texture coordinates of the vertex to be encoded according to the reconstructed geometric information corresponding to the three vertexes of the target triangle and the real texture coordinates of the vertexes on the first side, and obtains the predicted texture coordinates of the vertex to be encoded;

in step S43, the encoding end encodes the texture coordinates of the vertex to be encoded according to the difference between the actual texture coordinates of the vertex to be encoded and the predicted texture coordinates.

It should be noted that after obtaining the predicted texture coordinates of the vertex to be encoded, the difference between the predicted texture coordinates and the real texture coordinates may be obtained, and then the difference may be encoded based on the difference, alternatively, the difference may be directly encoded, or the difference may be processed first, and then the processed difference may be encoded, for example, the processing may be normalization processing; by encoding based on the difference value to thereby realize encoding of the vertices to be encoded, the number of bits of texture coordinate encoding can be reduced.

It should be noted that, in the embodiment of the present application, the obtaining manner of the difference may be obtained by subtracting the predicted texture coordinate from the actual texture coordinate, or may be obtained by subtracting the predicted texture coordinate from the actual texture coordinate, and the specific manner is what is adopted, so long as the encoding end and the decoding end understand to be consistent. The difference in the embodiments of the present application may also be referred to as a residual.

Optionally, in an embodiment of the present application, the implementation manner of predicting texture coordinates of a vertex to be encoded according to reconstructed geometric information corresponding to three vertices of the target triangle and real texture coordinates of vertices on the first edge by the encoding end to obtain predicted texture coordinates of the vertex to be encoded includes:

step S51, the encoding end obtains texture coordinates of projection points of the to-be-encoded vertexes on the first side according to geometric information corresponding to the vertexes of the target triangle and real texture coordinates of the vertexes on the first side;

it should be noted that, as shown in fig. 9, the edge NP is a selected edge in the edge set, and may be regarded as the first edge, the vertex N and the vertex P are two vertices of the first edge, the vertex C is a vertex to be encoded, the vertex N, the vertex P and the vertex C form the target triangle, the point X is a projection of the vertex C on the NP edge, the vertex O is an encoded point, and the triangle formed by the vertex O, the vertex N and the vertex P and the triangle formed by the vertex N, the vertex P and the vertex C form the NP edge. Based on fig. 9, optionally, the specific obtaining manner of the texture coordinate of the projected point of the vertex on the first edge in the embodiment of the present application is:

According to the formula:obtaining texture coordinates of projection points of the vertexes on the first edge;

wherein X is _uv Texture coordinates, N, of the projected points of the vertex C to be coded on the first side _uv Is the true texture coordinates of vertex N on the first side of the target triangle,a vector of texture coordinates from a vertex N on the first side of the target triangle to a projection point X of the vertex C to be coded on the first side; vectors of reconstructed geometric information corresponding to the vertexes N and P on the first side; />Vector of reconstructed geometric information corresponding to the vertex N to be coded C on the first edge; />And the vector of the reconstructed geometric information corresponding to the projection point X from the vertex N on the first side to the vertex C to be coded on the first side is obtained.

Step S52, the coding end obtains the predicted texture coordinates of the vertex to be coded according to the texture coordinates of the projection points;

optionally, the predicted texture coordinates of the vertex to be encoded include:

according to the formula:obtaining predicted texture coordinates of a vertex to be encoded;

wherein, pred _C For the predicted texture coordinates of the vertex C to be encoded, the first vector of the vectors of the texture coordinates of the vertex C to be encoded and the first edge corresponding to the first vertex O takes the value of O _uv The texture coordinates of a first vertex corresponding to the first edge of the target triangle, wherein the first vertex O is a pair of vertices of the first edge of the first triangle (namely, a vertex opposite to the first edge in the first triangle), and the first triangle and the target triangle have a common first edge; />A vector from a projection point X of the vertex C to be encoded on the first edge to texture coordinates of the vertex C to be encoded; /> And taking values for a second vector of the vectors of the texture coordinates of the vertex C to be coded and the first edge corresponding to the first vertex O.

It should be noted that, based on the above process, the predicted texture coordinates of the point to be encoded are obtained, and based on the predicted texture coordinates, the point to be encoded can be encoded; optionally, after encoding a point to be encoded based on one edge in an edge set, the encoding end adds a second edge in the target triangle to the edge set and deletes the first edge from the edge set, where the second edge is an edge in the target triangle that is not included in the edge set, so as to implement updating of the edge set.

In the embodiment of the present application, the residual error of the vertex to be encoded may be obtained while encoding, or all the residual errors may be obtained first, and then encoding of the residual errors may be performed uniformly.

It should be further noted that, in order to ensure consistency of texture coordinates, in an embodiment of the present application, the encoding end needs to adjust the value and the arrangement sequence of the partial texture coordinates according to the reconstructed first geometric information.

It should be noted that, in an embodiment of the present application, encoding the texture map is further included, specifically: for texture maps, a video encoder may be used directly to encode the frame-by-frame texture map, such as an encoder for efficient video coding (High Efficiency Video Coding, HEVC), a multi-function codec (VVC), or the like, to form a texture map sub-stream. Any video encoder may be used herein.

After each sub-code stream is obtained, each sub-code stream is mixed to form an output trellis-coded code stream.

In summary, for the encoding end, the geometric information, the connection relation, the UV coordinates and the texture map of the mesh are encoded respectively. The method mainly comprises the following steps:

1) In the lossy mode, the three-dimensional grid is optionally simplified, that is, the number of grid vertices and edges is reduced while maintaining the grid structure as much as possible, thereby reducing the data size of the three-dimensional grid.

2) And quantizing the geometric information of the simplified grid to reduce the distance between the vertexes, so that the subsequent geometric information compression based on the video is facilitated.

It should be noted that, by quantization, low-precision geometric information, high-precision geometric information, and information of supplemental points may be obtained. Wherein the low-precision geometric information is quantized three-dimensional coordinates; the high-precision geometric information is geometric information of the quantized low-precision grid, which is lost; the supplemental points refer to points that need additional processing, such as duplicate points where the coordinate positions overlap, etc., generated during quantization. For low-precision geometric information generated by quantization, a projection method similar to V-PCC is adopted to project the low-precision geometric information to a two-dimensional plane, and then video compression is carried out. And the high-precision geometric information, the supplementary points and the like can be encoded by adopting a processing method for special points in V-PCC, such as compression by using a raw patch method. And mixing the obtained subcode streams, preparing for further mixing the subcode streams with the connection relation code streams and other code streams to obtain a final output code stream. The lossless mode and the lossy mode are considered in the representation and the coding of the information with different precision and the complementary points, and particularly, lossless compression can be ensured while the coding bit cost is reduced.

3) And decoding the encoded geometric information code stream to reconstruct geometric information to be used as auxiliary information of connection relation encoding, and encoding the updated connection information after updating the connection relation of the original grid. And finally, mixing the obtained connection relation code stream with geometric information and other attribute information code streams needing to be encoded.

4) And reconstructing the UV coordinates by using the reconstructed geometric information and the UV coordinate information, and encoding the UV coordinates to form a UV coordinate subcode stream. The encoding method herein may use predictive encoding, i.e. encoding the residual between the predicted coordinates and the real coordinates after predicting the position of the UV coordinates to be encoded, to save bit overhead (the number of bits of the residual is smaller than the number of bits of the original UV coordinates).

5) And performing video coding on the texture map to form a texture map sub-code stream.

6) The obtained mixed streams of the sub-code streams are used as the output code stream of the encoder.

For example, as shown in fig. 10, the encoding framework of the video-based three-dimensional grid according to the embodiment of the present application, the overall encoding flow is: first, whether to sample the three-dimensional grid for simplification can be selected before quantization; then, the three-dimensional grid is quantized, thereby possibly generating three parts of low-precision geometric information, high-precision geometric information and supplementary point information; for low-precision geometric information, performing patch division in a projection mode, and generating patch sequence compression information (patch division information), a occupation map and a low-precision geometric figure by patch arrangement; for possible high-precision geometric information, a row patch arrangement mode can be adopted to generate a high-precision geometric figure; for the possible supplementary points, the geometric information of the supplementary points can be divided into a low-precision part and a high-precision part, and the low-precision part and the high-precision part are respectively arranged by a raw patch, and are independently encoded into a code stream, or the raw patch is added into a geometric figure; the coded patch sequence then compresses information, occupancy maps, geometry maps. And for the connection relation, decoding the encoded geometric information code stream to reconstruct the geometric information to be used as auxiliary information of the connection relation encoding. And after updating the connection relation of the original grid, coding the updated connection information to obtain a connection relation subcode stream. And for the UV coordinates, reconstructing the UV coordinates by using the reconstructed geometric information and the UV coordinate information, and encoding the UV coordinates to form a UV coordinate subcode stream. In some cases, such as when texture map slices are more trivial, the UV coordinates of the more complete regenerated slices may be selected and encoded. For texture maps, the texture maps may be directly video encoded to form texture map subcode streams. And finally, mixing the multiple paths of sub-code streams to obtain a final output code stream.

It should be noted that, the embodiment of the application provides a video-based three-dimensional grid coding framework, and for a decoding end, decoding is only needed according to the inverse process of the coding end; in the implementation of the application, the independent simple identification and coding are carried out particularly for the grids containing the geometric information repeated vertexes, and in the coding connection relationship process, the coding of the three-dimensional grids can be more efficient because the repeated vertexes are removed.

According to the encoding method provided by the embodiment of the application, the execution main body can be an encoding device. In the embodiment of the present application, an encoding method performed by an encoding device is taken as an example, and the encoding device provided in the embodiment of the present application is described.

As shown in fig. 11, an embodiment of the present application provides an encoding apparatus 1100, including:

a first obtaining module 1101, configured to decode and reconstruct the geometric information of the encoded target three-dimensional grid, and obtain reconstructed first geometric information;

the second obtaining module 1102 is configured to perform repeated vertex screening on the first geometric information to obtain repeated vertex information, where the repeated vertex is a vertex other than a first vertex in multiple vertices with the same position coordinates, and the first vertex is one vertex in multiple vertices with the same position coordinates;

A third obtaining module 1103, configured to reorder the first connection relationship of the target three-dimensional mesh according to the repeated vertex information, and obtain a reordered second connection relationship;

a first encoding module 1104, configured to encode the second connection relationship.

Optionally, the method for obtaining the repeated vertex information includes:

acquiring repeated vertexes in the first geometric information, and recording indexes of the repeated vertexes to form repeated vertex information;

Optionally, the third obtaining module 1103 includes:

the first acquisition unit is used for rearranging the vertex texture coordinates of the target three-dimensional grid according to the indexes of the repeated vertices and acquiring the reordered texture coordinate information;

and the second acquisition unit is used for adjusting the first connection relation of the target three-dimensional grid according to the reordered texture coordinate information and the repeated vertex information to acquire a second connection relation after reordering.

Optionally, the first obtaining unit is configured to:

and sequentially moving texture coordinates corresponding to the repeated vertexes to the tail part of a list corresponding to the texture coordinates according to the sequence of the repeated vertexes in the first geometric information, so as to form reordered texture coordinate information.

Optionally, the second obtaining unit is configured to:

modifying a texture coordinate index part of a first connection relation of the target three-dimensional grid according to the reordered texture coordinate information;

traversing the geometric index part of the first connection relation of the target three-dimensional grid, replacing the index of the repeated vertexes of the first connection relation of the target three-dimensional grid with the index of the first vertexes according to the repeated vertex information, and obtaining the second connection relation after reordering.

Optionally, the apparatus further comprises:

the first query module is used for querying whether repeated vertexes exist for three vertexes of each triangular surface patch according to repeated vertex information in the process of encoding the second connection relation;

the identification module is used for identifying the repeated vertexes under the condition that the repeated vertexes exist;

and the fourth acquisition module is used for carrying out coding according to the identification of the repeated vertex after the coding of the second connection relation is completed, so as to acquire the repeated vertex identification code stream.

Optionally, after the first obtaining module 1101 decodes and reconstructs the geometric information of the encoded target three-dimensional grid, the method further includes:

And the adjusting module is used for adjusting the value and the arrangement sequence of the partial texture coordinates according to the reconstructed first geometric information.

Optionally, the apparatus further comprises:

the first recovery module is used for recovering repeated vertexes in the second connection relation with second geometric information according to the sequence of the repeated vertexes in the reordered texture coordinate information, wherein the second geometric information is grid geometric information which does not comprise the repeated vertexes;

the first recording module is used for traversing the second connection relation of the recovered repeated vertexes and recording the traversing sequence;

and the second coding module is used for coding the texture coordinates of the vertexes according to the traversing sequence.

Optionally, before the third obtaining module 1103 reorders the first connection relationship of the target three-dimensional mesh according to the repeated vertex information, the method further includes:

and the adjusting module is used for adjusting the part with the change of the first connection relation of the target three-dimensional grid according to the reconstructed first geometric information.

The embodiment of the device corresponds to the embodiment of the encoding method, and each implementation process and implementation manner of the embodiment of the method can be applied to the embodiment of the device, and the same technical effects can be achieved.

The embodiment of the application also provides encoding equipment, which comprises a processor and a communication interface, wherein the processor is used for decoding and reconstructing the geometric information of the encoded target three-dimensional grid to obtain reconstructed first geometric information; the first geometric information is subjected to repeated vertex screening, repeated vertex information is obtained, the repeated vertexes are among the vertexes with the same position coordinates, except for a first vertex, and the first vertex is one vertex among the vertexes with the same position coordinates; according to the repeated vertex information, reordering the first connection relation of the target three-dimensional grid, and acquiring a reordered second connection relation; and encoding the second connection relation.

Optionally, the processor is configured to:

the encoding end rearranges vertex texture coordinates of the target three-dimensional grid according to indexes of repeated vertices, and acquires reordered texture coordinate information;

And the encoding end adjusts the first connection relation of the target three-dimensional grid according to the reordered texture coordinate information and the repeated vertex information, and acquires a second connection relation after reordering.

Optionally, the processor is configured to:

and the coding end sequentially moves the texture coordinates corresponding to the repeated vertexes to the tail part of the list corresponding to the texture coordinates according to the sequence of the repeated vertexes in the first geometric information, so as to form reordered texture coordinate information.

Optionally, the processor is configured to:

the coding end modifies the texture coordinate index part of the first connection relation of the target three-dimensional grid according to the reordered texture coordinate information;

the encoding end traverses the geometric index part of the first connection relation of the target three-dimensional grid, replaces the index of the repeated vertex of the first connection relation of the target three-dimensional grid with the index of the first vertex according to the repeated vertex information, and acquires the second connection relation after reordering.

Optionally, the processor is further configured to:

the coding end adjusts the value and the arrangement sequence of the partial texture coordinates according to the reconstructed first geometric information.

Optionally, the processor is further configured to:

the encoding end recovers the repeated vertexes in the second connection relation with the second geometric information according to the sequence of the repeated vertexes in the reordered texture coordinate information, wherein the second geometric information is grid geometric information which does not comprise the repeated vertexes;

Optionally, the processor is further configured to:

and the coding end adjusts the part with the changed first connection relation of the target three-dimensional grid according to the reconstructed first geometric information.

Specifically, the embodiment of the present application further provides an encoding apparatus, as shown in fig. 12, the encoding apparatus 1200 includes: a processor 1201, a network interface 1202, and a memory 1203. The network interface 1202 is, for example, a common public radio interface (common public radio interface, CPRI).

Specifically, the encoding apparatus 1200 of the embodiment of the present application further includes: instructions or programs stored in the memory 1203 and capable of being executed by the processor 1201, the processor 1201 calls the instructions or programs in the memory 1203 to execute the method executed by each module shown in fig. 11, and achieve the same technical effects, so that repetition is avoided and thus a detailed description is omitted.

As shown in fig. 13, the embodiment of the present application further provides a decoding method, which corresponds to an implementation process of the encoding end, and includes:

step 1301, a decoding end decodes and reconstructs first geometric information according to the code stream corresponding to the obtained target three-dimensional grid;

step 1302, the decoding end performs repeated vertex filtering on the first geometric information to obtain repeated vertex information, where the repeated vertex is a vertex except a first vertex in multiple vertices with the same position coordinates, and the first vertex is one vertex in multiple vertices with the same position coordinates;

and step 1303, the decoding end decodes the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the method for obtaining the repeated vertex information includes:

the decoding end acquires repeated vertexes in the first geometric information and records indexes of the repeated vertexes to form repeated vertex information;

Optionally, the method further comprises:

in the process of decoding the connection relation code stream, the decoding end inquires whether repeated vertexes exist for each triangular patch according to the repeated vertex information;

the decoding end decodes the index of the repeated vertex from the repeated identification code stream under the condition that the repeated vertex exists.

Optionally, the method further comprises:

the decoding end restores the second geometric information and the connection relation obtained by decoding according to the indexes of the repeated vertexes, wherein the second geometric information is grid geometric information which does not comprise the repeated vertexes;

the decoding end traverses the connection relation of the recovered repeated vertexes and records the traversing sequence;

and the decoding end decodes the texture coordinate code stream of the vertex according to the traversing sequence.

Optionally, the video-based three-dimensional grid geometry information decoding process in the embodiment of the present application includes: decomposing the code stream into a latch information subcode stream, a duty bit stream and a geometric figure code stream; then, decoding the three sub-code streams respectively to obtain patch information, a occupation map and a geometric map; finally, the geometry information is reconstructed using the patch information, the occupancy map, and the geometry map. Specifically, as shown in fig. 14, the most critical specific process of geometric information reconstruction is:

Step S111, acquiring a 2D patch;

it should be noted that, acquiring the 2D patch refers to dividing the occupation information and the depth information of the 2D patch from the occupation map and the geometric map by using the patch information. The Patch information comprises the position and the size of the bounding box of each 2D Patch in the occupation map and the low-precision geometric map, and the occupation information and the low-precision geometric information of the 2D Patch can be directly obtained by using the Patch information, the occupation map and the low-precision geometric map. And for the high-precision geometric information, using the vertex distribution of the low-precision geometric figure to correspond the high-precision geometric information arrangement value in the high-precision geometric information area with the low-precision geometric figure vertex, and separating xyz three-dimensional geometric information from the high-precision geometric information according to the preset arrangement rule used by the encoding end, thereby obtaining the high-precision geometric information. For the geometric information of the supplementary point, the low-precision geometric information and the high-precision geometric information of the supplementary point can be obtained by directly decoding the low-precision raw patch and the high-precision raw patch of the supplementary point.

Step S112, reconstructing a 3D patch;

it should be noted that reconstructing the 3D patch refers to reconstructing vertices in the 2D patch into the low-precision 3D patch by using the occupancy information and the low-precision geometric information in the 2D patch. The occupation information of the 2D patch comprises the position of the vertex relative to the origin of coordinates in a local coordinate system of the patch projection plane, and the depth information comprises the depth value of the vertex in the normal direction of the projection plane. Thus, the 2D patch can be reconstructed into a low-precision 3D patch in a local coordinate system using the occupancy information and the depth information.

Step S113, reconstructing a low-precision geometric model;

it should be noted that reconstructing the low-precision geometric model refers to reconstructing the entire low-precision three-dimensional geometric model by using the reconstructed low-precision 3D patch. The Patch information comprises a conversion relation of converting the 3D Patch from a local coordinate system to a global coordinate system of the three-dimensional geometric model, and all the 3D patches are converted into the global coordinate system by utilizing the coordinate conversion relation, so that the low-precision three-dimensional geometric model is obtained. In addition, for the supplementary points, the geometric information in the low-precision raw patch is directly utilized to obtain low-precision coordinate values of the supplementary points under the global coordinate system, so that a complete low-precision three-dimensional geometric model is obtained.

Step S114, reconstructing a high-precision geometric model;

the reconstruction of the high-precision geometric model refers to a process of reconstructing the high-precision geometric model by using high-precision geometric information on the basis of the low-precision geometric model. In the process of acquiring the 2D patch, the high-precision geometric information and the low-precision geometric information are corresponding, and the high-precision three-dimensional coordinate of the vertex can be reconstructed according to the high-precision geometric information and the low-precision geometric information of the vertex. According to the application requirements, high-precision three-dimensional coordinates of all vertexes can be selected to be reconstructed, and high-precision three-dimensional coordinates of partial vertexes can also be reconstructed. High precision three-dimensional coordinates (x _r ,y _r ,z _r ) As shown in the formula thirty-eight to the formula forty:

the formula thirty-eighth: x is x _r ＝f ₃ (x _l ,x _h ,QP _x )；

The formula thirty-nine: y is _r ＝f ₃ (y _l ,y _h ,QP _y )；

Formula forty: z _r ＝f ₃ (z _l ,z _h ,QP _z )；

f ₃ The function is a reconstruction function, the calculation process of the reconstruction function corresponds to the calculation process of the quantization function at the encoding end, and various implementations are realizedMode(s). If f ₁ The function adopts the realization modes from the formula twenty to the formula twenty-five, and the realization mode of the reconstruction function is as shown in the formula forty-eight to the formula forty-three:

formula forty-one: x is x _r ＝x _l *QP _x +x _h ；

Formula forty-two: y is _r ＝y _l *QP _y +y _h ；

The formula forty-three: z _r ＝z _l *QP _z +z _h ；

If f ₁ The function adopts the implementation modes from the formula twenty-six to the formula thirty-one, and the implementation mode of the reconstruction function is as shown in the formula forty-four to the formula forty-six:

the formula forty-four: x is x _r ＝(x _l ＜＜log ₂ QP _x )|x _h ；

Forty-five of the formula: y is _r ＝(y _l ＜＜log ₂ QP _y )|y _h ；

Formula forty-six: z _r ＝(z _l ＜＜log ₂ QP _z )|z _h 。

It should be noted that, in the embodiment of the present application, when decoding the connection relationship, the reconstructed geometric information is first needed to find the repeated vertex, remove the repeated vertex, and record the vertex index of the repeated point to form the repeated vertex information. Specifically, the process of decoding the connection relationship may specifically include:

(1) The vertices of the original triangle are decoded and the edges of the original triangle are stored in the edge set. The edge τ is fetched from the edge set according to the criteria used in the encoder. If the codeword to be decoded is a vertex index, the vertex is directly decoded and used as a counter vertex v. A newly decoded triangle is formed with the vertex v and the edge τ, and two edges of the triangle except the edge τ are added to the set of edges, with the edge τ in the set removed with some rule, such as removing the edge τ according to the criteria of the top of the queue. The next edge is fetched according to a rule and continues to decode the edge adjacency triangle, e.g., the edge at the top of the queue may be fetched as the rule.

(2) If the codeword to be decoded is not a vertex index, then the spatial extent to vertex v is decoded and determined. For example decoding two concentric sphere radii R of τmidpoint _min ,R _max Traversing all vertices in the inter-concentric sphere range, and screening out all vertices of a new triangle forming an angle with the decoded triangle smaller than a certain angle with the edge τ.

(3) And for the rest vertexes in the space range of the vertex v, sorting the vertexes according to the sorting criterion same as that of the encoding end. And decoding a serial number corresponding to the vertex v in the triangle to be decoded, looking up a table to obtain the vertex v, and constructing the decoded triangle. Two edges of the new decoded triangle except for edge τ are added to the set of edges, and edge τ in the set is removed with some rule, such as removing edge τ according to the criteria of the top of the queue. The next edge is fetched according to a rule and continues to decode the edge adjacency triangle, e.g., the edge at the top of the queue may be fetched as the rule.

The decoding process is iterated on the stream of each slice until the triangles in each slice are decoded. The tiles are eventually consolidated into a complete grid.

In the process of decoding the connection relationship, for each triangular patch, according to the repetitive point comparison table, if the vertex of the current triangular patch is the repetitive vertex, the current repetitive vertex index is decoded from the repetitive identification code stream. Specifically, the duplicate vertex identification is decoded and placed in an array. And then removing the repeated vertex mark from the array to form the repeated vertex index according to the repeated vertex information if the current vertex is the repeated vertex.

It should be noted that, the decoding process of the UV coordinates of the three-dimensional grid is the inverse of the encoding process, and in a specific embodiment of the present application, the coordinate prediction residual is entropy decoded first. And recovering repeated vertexes in the geometric information and connectivity relation, iterating the connection relation, recording the connection sequence of the points, and predicting the UV coordinates of the vertexes according to the sequence. The prediction process predicts the current point according to the decoded point UV coordinates and the encoding end prediction rule by utilizing the projection mapping relation from three dimensions to two dimensions of the triangle to obtain predicted UV coordinates. And adding the residual error value obtained by entropy decoding to the predicted coordinate to obtain the position of the UV coordinate to be decoded.

Here, the UV coordinates of the initial triangle are not coded by prediction, but are directly coded, and after the decoding end decodes the UV coordinates of the triangle, the initial triangle starts to traverse the UV coordinates of the vertices of other triangles.

For texture map decoding, in an embodiment of the present application, a video decoder is directly used to decode the texture map, so as to obtain a frame-by-frame texture map, where the file format of the texture map is not emphasized, and the format may be jpg, png, etc.

In summary, the three-dimensional trellis decoding process can be divided into:

1. and decoding each path of sub-code stream after the decoding, wherein the video decoder is used for decoding the occupancy map, the geometric map and the texture map, and the decoder corresponding to the encoding end encoding method is used for decoding the connection relation and the UV coordinate sub-code stream.

2. For geometric information, firstly, a code stream is decomposed into a patch information sub-code stream, a occupation bit map code stream and a geometric figure code stream; then, decoding the three sub-code streams respectively to obtain patch information, a occupation map and a geometric map; finally, the geometry information is reconstructed using the patch information, the occupancy map, and the geometry map.

3. And for the connection relation, decoding the connection relation subcode stream, and reconstructing the connection relation.

4. And decoding the connection relation subcode stream for the UV coordinates, and reconstructing the UV coordinates.

5. And for the texture map, decoding the texture map sub-code stream to obtain a frame-by-frame texture map.

It should be noted that, the decoding process of the embodiment of the present application is an inverse process of encoding, and the decoding block diagram is shown in fig. 15, where first, the code stream is decomposed into a latch information subcode stream, a bitmap subcode stream, a geometry map subcode stream, a connection relation subcode stream, a UV coordinate subcode stream, and a texture map subcode stream, and then decoded respectively; the geometric information of the low-precision grid can be reconstructed by using the occupation map and the low-precision geometric map, and the geometric information of the high-precision grid can be reconstructed by using the occupation map, the low-precision geometric map and the high-precision geometric map; decoding the connection relation of the grid by using the connection relation subcode stream; decoding UV coordinates of the grid from the UV coordinate subcode stream; decoding a frame-by-frame texture map from the texture map sub-code stream; finally, reconstructing the three-dimensional grid by using the reconstructed geometric information, connection information, UV coordinate information, texture map and other information.

It should be noted that, the embodiment of the present application is an opposite-end method embodiment corresponding to the foregoing encoding method embodiment, the decoding process is an inverse process of encoding, and all the foregoing implementation manners of the encoding side are applicable to the decoding-end embodiment, so that the same technical effects can be achieved, which is not repeated herein.

As shown in fig. 16, an embodiment of the present application further provides a decoding apparatus 1600, including:

a reconstruction module 1601, configured to decode and reconstruct the first geometric information according to the obtained code stream corresponding to the target three-dimensional grid;

a fifth obtaining module 1602, configured to perform repeated vertex filtering on the first geometric information to obtain repeated vertex information, where the repeated vertex is a vertex other than a first vertex among multiple vertices with the same position coordinates, and the first vertex is one of multiple vertices with the same position coordinates;

and the decoding module 1603 is configured to decode the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the method for obtaining the repeated vertex information includes:

Optionally, the apparatus further comprises:

the second query module is used for querying whether repeated vertexes exist for each triangular patch according to the repeated vertex information in the process of decoding the connection relation code stream;

and the second decoding module is used for decoding the index of the repeated vertex from the repeated identification code stream under the condition that the repeated vertex exists at the decoding end.

Optionally, the apparatus further comprises:

the second recovery module is used for recovering the second geometric information and the connection relation obtained by decoding according to the indexes of the repeated vertexes, wherein the second geometric information is grid geometric information which does not comprise the repeated vertexes;

the second recording module is used for traversing the connection relation of the recovered repeated vertexes and recording the traversing sequence;

and the third decoding module is used for decoding the texture coordinate code stream of the vertex according to the traversing sequence.

It should be noted that, the embodiment of the apparatus is an apparatus corresponding to the above method, and all implementation manners in the embodiment of the method are applicable to the embodiment of the apparatus, so that the same technical effects can be achieved, which is not described herein again.

Preferably, the embodiment of the present application further provides a decoding device, which includes a processor, a memory, and a program or an instruction stored in the memory and capable of running on the processor, where the program or the instruction is executed by the processor to implement each process of the decoding method embodiment described above, and the same technical effects can be achieved, so that repetition is avoided, and details are not repeated here.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements the respective processes of the decoding method embodiment described above, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here.

Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

The embodiment of the application also provides decoding equipment, which comprises a processor and a communication interface, wherein the processor is used for decoding and reconstructing first geometric information according to the code stream corresponding to the obtained target three-dimensional grid; the first geometric information is subjected to repeated vertex screening, repeated vertex information is obtained, the repeated vertexes are among the vertexes with the same position coordinates, except for a first vertex, and the first vertex is one vertex among the vertexes with the same position coordinates; and decoding the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

Optionally, the processor is configured to:

in the process of decoding the connection relation code stream, inquiring whether repeated vertexes exist for each triangular patch according to the repeated vertex information;

in the presence of duplicate vertices, an index of the duplicate vertices is decoded from the duplicate identification code stream.

Optionally, the processor is configured to:

recovering the second geometric information and the connection relation obtained by decoding according to the indexes of the repeated vertexes, wherein the second geometric information is grid geometric information which does not comprise the repeated vertexes;

traversing the connection relation of the recovered repeated vertexes, and recording the traversing sequence;

and decoding the texture coordinate code stream of the vertex according to the traversing sequence.

The decoding device embodiment corresponds to the decoding method embodiment, and each implementation process and implementation manner of the method embodiment can be applied to the decoding device embodiment, and the same technical effect can be achieved.

Specifically, the embodiment of the application also provides decoding equipment. Specifically, the structure of the decoding device is shown in fig. 12, and will not be described here again. Specifically, the decoding device of the embodiment of the present application further includes: instructions or programs stored in the memory and executable on the processor call the instructions or programs in the memory to perform the method performed by each module shown in fig. 16, and achieve the same technical effects, so repetition is avoided and will not be described herein.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements each process of the decoding method embodiment described above, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here.

Wherein the processor is a processor in the decoding device described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc.

Optionally, as shown in fig. 17, the embodiment of the present application further provides a communication device 1700, including a processor 1701 and a memory 1702, where the memory 1702 stores a program or an instruction that can be executed on the processor 1701, for example, when the communication device 1700 is an encoding device, the program or the instruction implements the steps of the foregoing encoding method embodiment when executed by the processor 1701, and the same technical effects can be achieved. When the communication device 1700 is a decoding device, the program or the instructions implement the steps of the decoding method embodiment described above when executed by the processor 1701, and the same technical effects can be achieved, so that repetition is avoided and no further description is given here.

The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or instructions, the processes of the above coding method or decoding method embodiment can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, or the like.

The embodiments of the present application further provide a computer program/program product stored in a storage medium, where the computer program/program product is executed by at least one processor to implement each process of the foregoing encoding method or decoding method embodiment, and achieve the same technical effects, and are not repeated herein.

The embodiment of the application also provides a communication system, which at least comprises: an encoding device operable to perform the steps of the encoding method as described above, and a decoding device operable to perform the steps of the decoding method as described above. And the same technical effects can be achieved, and in order to avoid repetition, the description is omitted here.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims

1. A method of encoding, comprising:

the encoding end encodes the second connection relation.

2. The method according to claim 1, wherein the method for obtaining the repeated vertex information includes:

3. The method according to claim 1, wherein the encoding end performs reordering of the first connection relationship of the target three-dimensional mesh according to the repeated vertex information, and obtains a reordered second connection relationship, including:

4. The method according to claim 3, wherein the encoding end reorders vertex texture coordinates of the target three-dimensional mesh according to indexes of repeated vertices, and obtains reordered texture coordinate information, including:

5. The method of claim 3, wherein the encoding end adjusts the first connection relationship of the target three-dimensional mesh according to the reordered texture coordinate information and the repeated vertex information, and obtains the second connection relationship after reordering, including:

6. The method as recited in claim 1, further comprising:

7. The method according to claim 1, wherein after the encoding end decodes and reconstructs the encoded geometric information of the target three-dimensional grid, the method further comprises:

8. A method according to claim 3, further comprising:

9. The method according to claim 1, wherein before the encoding end reorders the first connection relation of the target three-dimensional mesh according to the repeated vertex information and obtains the reordered second connection relation, the method further comprises:

10. A decoding method, comprising:

and the decoding end decodes the connection relation code stream of the target three-dimensional grid according to the repeated vertex information.

11. The method according to claim 10, wherein the method for obtaining the repeated vertex information includes:

12. The method as recited in claim 10, further comprising:

13. The method as recited in claim 10, further comprising:

14. An encoding device, comprising:

and the first coding module is used for coding the second connection relation.

15. The apparatus of claim 14, wherein the means for obtaining the duplicate vertex information comprises:

16. The apparatus of claim 14, wherein the third acquisition module comprises:

17. The apparatus of claim 16, wherein the first acquisition unit is configured to:

18. The apparatus of claim 16, wherein the second acquisition unit is configured to:

19. The apparatus as recited in claim 14, further comprising:

20. The apparatus of claim 14, wherein after the first acquisition module decodes and reconstructs the encoded geometric information of the target three-dimensional grid, the first acquisition module further comprises:

21. The apparatus as recited in claim 16, further comprising:

22. The apparatus of claim 14, wherein before the third obtaining module reorders the first connection relation of the target three-dimensional mesh according to the repeated vertex information and obtains the reordered second connection relation, further comprising:

23. An encoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the encoding method according to any one of claims 1 to 9.

24. A decoding apparatus, comprising:

25. A decoding device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the decoding method according to any one of claims 10 to 13.

26. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the encoding method according to any one of claims 1 to 9 or the steps of the decoding method according to any one of claims 10 to 13.