WO2023238867A1 - 符号化装置、復号装置、符号化方法及び復号方法 - Google Patents

符号化装置、復号装置、符号化方法及び復号方法 Download PDF

Info

Publication number
WO2023238867A1
WO2023238867A1 PCT/JP2023/021028 JP2023021028W WO2023238867A1 WO 2023238867 A1 WO2023238867 A1 WO 2023238867A1 JP 2023021028 W JP2023021028 W JP 2023021028W WO 2023238867 A1 WO2023238867 A1 WO 2023238867A1
Authority
WO
WIPO (PCT)
Prior art keywords
triangle
vertex
encoding
information
decoding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/021028
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
敏康 杉尾
賀敬 井口
孝啓 西
チョン スン リム
ヂァン ウー
ハン ブン テオ
ケン リン ロイ
チャン ディーン ハン
ジョージ ナダー
ファーマン デュマノブ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Corp of America
Original Assignee
Panasonic Intellectual Property Corp of America
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Corp of America filed Critical Panasonic Intellectual Property Corp of America
Priority to JP2024526458A priority Critical patent/JP7723199B2/ja
Priority to EP23819847.7A priority patent/EP4538977A4/en
Priority to CN202380043575.9A priority patent/CN119301642A/zh
Priority to KR1020247038325A priority patent/KR20250019630A/ko
Publication of WO2023238867A1 publication Critical patent/WO2023238867A1/ja
Priority to MX2024014355A priority patent/MX2024014355A/es
Priority to US18/965,181 priority patent/US20250095214A1/en
Anticipated expiration legal-status Critical
Priority to JP2025128166A priority patent/JP2025166019A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame

Definitions

  • the present disclosure relates to an encoding device and the like.
  • Patent Document 1 proposes a method and apparatus for encoding and decoding three-dimensional mesh data.
  • the present disclosure aims to improve three-dimensional data encoding processing and the like.
  • An encoding device includes a memory and a circuit that can access the memory, and in operation, when the first triangle satisfies a condition, the circuit uses the first triangle to Selecting a mode for encoding the encoding target vertex from a first mode set including a normal mode for encoding the encoding target vertex of two triangles, and when the first triangle does not satisfy the condition, The mode is selected from a second mode set including an exception mode in which the encoding target vertex is encoded using a third triangle different from the first triangle and the second triangle, and the encoding target is encoded according to the mode. Encode the vertices.
  • the present disclosure can contribute to improvements in three-dimensional data encoding processing, etc.
  • FIG. 2 is a conceptual diagram showing a three-dimensional mesh according to an embodiment.
  • FIG. 2 is a conceptual diagram showing basic elements of a three-dimensional mesh according to an embodiment.
  • FIG. 2 is a conceptual diagram showing mapping according to an embodiment.
  • 1 is a block diagram showing a configuration example of an encoding/decoding system according to an embodiment.
  • FIG. 1 is a block diagram illustrating a configuration example of an encoding device according to an embodiment. It is a block diagram showing another example of composition of an encoding device concerning an embodiment.
  • FIG. 1 is a block diagram showing a configuration example of a decoding device according to an embodiment. It is a block diagram showing another example of composition of a decoding device concerning an embodiment.
  • FIG. 2 is a conceptual diagram showing a configuration example of a bitstream according to an embodiment.
  • FIG. 3 is a conceptual diagram illustrating another configuration example of a bitstream according to an embodiment.
  • FIG. 7 is a conceptual diagram showing yet another configuration example of a bitstream according to the embodiment.
  • FIG. 1 is a block diagram showing a specific example of an encoding/decoding system according to an embodiment.
  • FIG. 2 is a conceptual diagram showing a configuration example of point cloud data according to an embodiment.
  • FIG. 2 is a conceptual diagram showing an example of a data file of point cloud data according to an embodiment.
  • FIG. 2 is a conceptual diagram showing a configuration example of mesh data according to an embodiment.
  • FIG. 2 is a conceptual diagram showing an example of a data file of mesh data according to an embodiment.
  • FIG. 1 is a block diagram showing a specific example of an encoding/decoding system according to an embodiment.
  • FIG. 2 is a conceptual diagram showing a configuration example of point cloud data according to an embodiment
  • FIG. 3 is a conceptual diagram showing types of three-dimensional data according to the embodiment.
  • FIG. 2 is a block diagram showing a configuration example of a three-dimensional data encoder according to an embodiment.
  • FIG. 2 is a block diagram showing a configuration example of a three-dimensional data decoder according to an embodiment. It is a block diagram showing another example of composition of a three-dimensional data encoder concerning an embodiment. It is a block diagram showing another example of composition of a three-dimensional data decoder concerning an embodiment.
  • FIG. 3 is a conceptual diagram showing a specific example of encoding processing according to the embodiment.
  • FIG. 3 is a conceptual diagram showing a specific example of decoding processing according to the embodiment.
  • FIG. 2 is a block diagram illustrating an implementation example of an encoding device according to an embodiment.
  • FIG. 2 is a block diagram illustrating an implementation example of a decoding device according to an embodiment.
  • FIG. 2 is a conceptual diagram showing a parallelogram prediction method according to a reference example.
  • FIG. 2 is a conceptual diagram showing a polynomial prediction method according to a reference example. It is a flow chart which shows an example of encoding processing concerning a 1st aspect. 2 is a flowchart showing a specific example of encoding processing according to the first aspect. It is a flowchart which shows another example of encoding processing concerning a 1st aspect. It is a conceptual diagram which shows the example of three angles based on a 1st aspect. It is a conceptual diagram which shows another example of three angles based on a 1st aspect.
  • FIG. 7 is a conceptual diagram showing still another example of three angles according to the first aspect.
  • FIG. 7 is a syntax diagram showing an example of the syntax structure of vertex information for a plurality of vertices according to the first aspect.
  • FIG. 7 is a syntax diagram showing an example of the syntax structure of vertex information for one vertex according to the first aspect.
  • FIG. 7 is a syntax diagram showing another example of the syntax structure of vertex information for one vertex according to the first aspect.
  • FIG. 3 is a table diagram showing an example of a table used for angle prediction.
  • FIG. 7 is a table diagram showing another example of a table used for angle prediction.
  • FIG. 6 is a conceptual diagram showing an example of prediction using an angle with respect to a vertex of a previous triangle.
  • FIG. 7 is a syntax diagram showing an example of the syntax structure of vertex information for a plurality of vertices according to the first aspect.
  • FIG. 7 is a syntax diagram showing an example of the syntax structure of vertex information for
  • FIG. 7 is a conceptual diagram showing an example of prediction using angles with respect to vertices of a previous three-dimensional mesh.
  • FIG. 2 is a block diagram showing a configuration example of an encoding device according to a first aspect. It is a flow chart which shows an example of decoding processing concerning a 1st aspect. 12 is a flowchart showing a specific example of decoding processing according to the first aspect. It is a flowchart which shows another example of decoding processing concerning a 1st aspect.
  • FIG. 3 is a table diagram showing an example of a lookup table for a sine function and a cosine function.
  • FIG. 2 is a block diagram showing a configuration example of a decoding device according to a first aspect.
  • FIG. 12 is a flowchart illustrating an example of encoding processing according to a second aspect. It is a conceptual diagram which shows the example of the mesh based on a 2nd aspect.
  • FIG. 3 is a conceptual diagram showing an example of a fragmented mesh.
  • FIG. 2 is a conceptual diagram showing an example of a mesh with overlapping vertices.
  • FIG. 3 is a conceptual diagram illustrating an example of exception handling using a previously encoded vertex set.
  • 12 is a flowchart showing a specific example of encoding processing according to the second aspect. It is a block diagram showing an example of composition of an encoding device concerning a 2nd aspect. 12 is a flowchart illustrating an example of decoding processing according to a second aspect.
  • FIG. 12 is a flowchart showing a specific example of decoding processing according to a second aspect. It is a block diagram showing an example of composition of a decoding device concerning a 2nd aspect. 7 is a flowchart illustrating an example of exception handling according to a second aspect. It is a flowchart which shows another example of encoding processing concerning a 2nd aspect. It is a flowchart which shows another example of decoding processing concerning a 2nd aspect.
  • FIG. 2 is a conceptual diagram showing an example of the position of parameters in a bitstream.
  • FIG. 7 is a conceptual diagram showing another example of the position of parameters in a bitstream.
  • 3 is a flowchart illustrating an example of basic encoding processing according to the embodiment.
  • FIG. 3 is a flowchart illustrating an example of basic decoding processing according to the embodiment. It is a block diagram showing still another example of composition of an encoding device concerning an embodiment. It is a block diagram showing still another example of composition of a decoding device concerning an embodiment.
  • a three-dimensional mesh is composed of vertex information indicating the position of each of a plurality of vertices in a three-dimensional space, connection information indicating a connection relationship between the plurality of vertices, and attribute information indicating an attribute of each vertex or each face.
  • Each surface is constructed according to the connection relationship of a plurality of vertices.
  • Various computer graphics images can be expressed using such a three-dimensional mesh.
  • the encoding device predicts the position information of the vertex to be encoded using the position information of the encoded vertex, and encodes the difference with respect to the prediction. , reduce the amount of code. More specifically, for example, the encoding device refers to the first triangle as a reference triangle and encodes the vertices of the second triangle adjacent to the first triangle, thereby reducing the amount of code.
  • the first triangle is not suitable as a reference triangle.
  • the vertices of the first triangle may not be encoded or the sides of the first triangle may not have connectivity. Even if the vertices of the second triangle are encoded with reference to such a first triangle, it may become difficult to decode the vertices of the second triangle, or the amount of code may not be reduced.
  • the encoding device of Example 1 includes a memory and a circuit that can access the memory, and in operation, when the first triangle satisfies the condition, the circuit uses the first triangle to generate a second triangle.
  • a mode for encoding the encoding target vertex is selected from a first mode set including a normal mode for encoding the encoding target vertex, and if the first triangle does not satisfy the condition, the first mode set includes a normal mode for encoding the encoding target vertex.
  • the mode is selected from a second mode set including an exception mode in which the encoding target vertex is encoded using a third triangle different from the first triangle and the second triangle, and the encoding target vertex is encoded according to the mode. encode.
  • the first triangle does not satisfy the condition, it may be possible to encode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to encode the vertices of the second triangle using a reference triangle suitable for encoding.
  • the encoding device of Example 2 may be the encoding device of Example 1, in which the third triangle is a triangle that shares a side with the first triangle.
  • the encoding device of Example 3 may be the encoding device of Example 1 or 2, wherein the third triangle is a triangle that shares a side with the second triangle. .
  • the encoding device of Example 4 is the encoding device of any one of Examples 1 to 3, in which the third triangle is a triangle that shares an apex with the first triangle. It's okay.
  • the encoding device of Example 5 is the encoding device of any one of Examples 1 to 4, wherein the third triangle is a triangle in which each vertex has been encoded. Good too.
  • the encoding device of Example 6 is the encoding device of any one of Examples 1 to 5, wherein the third triangle is a triangle in which each side has connectivity. There may be.
  • the encoding device of Example 7 may be the encoding device of any of Examples 1 to 4, in which the third triangle is a triangle having a dummy apex.
  • the encoding device of Example 8 is the encoding device of any one of Examples 1 to 7, wherein the exception mode includes a plurality of submodes, and the circuit further includes the exception mode as the mode. is selected, the encoding device may select a submode for encoding the encoding target vertex from among the plurality of submodes.
  • This may make it possible to encode the encoding target vertex according to a submode selected from a plurality of submodes in the exception mode. Therefore, it may be possible to encode a target vertex to be encoded according to an exception handling method that is adaptively selected from among a plurality of exception handling methods.
  • the encoding device of Example 9 may be the encoding device of Example 8, in which the circuit encodes a parameter indicating the submode.
  • the encoding device of Example 10 may be the encoding device of Example 8 or Example 9, in which the circuit determines the third triangle according to the submode.
  • the encoding device of Example 11 is the encoding device of any of Examples 8 to 10, wherein the circuit determines the vertex of the third triangle according to the submode. It's okay.
  • the encoding device of Example 12 is the encoding device of any one of Examples 1 to 11, wherein the condition includes that each vertex of the first triangle has been encoded. It may be.
  • the encoding device of Example 13 is the encoding device of any of Examples 1 to 12, wherein the condition includes that each side of the first triangle has connectivity. It may also be a conversion device.
  • the encoding device of Example 14 is the encoding device of any one of Examples 1 to 13, in which the circuit encodes the third triangle when encoding the encoding target vertex according to the exception mode.
  • the encoding apparatus may encode an angle indicating the encoding target vertex using the encoder.
  • This may make it possible to prevent the code amount from becoming too large. That is, it may be possible to suppress the worst case code amount.
  • the decoding device of Example 15 includes a memory and a circuit that can access the memory, and in operation, when the first triangle satisfies a condition, the circuit uses the first triangle to generate a second triangle. If a mode for decoding the target vertex is selected from a first mode set including a normal mode for decoding the target vertex, and the first triangle does not satisfy the conditions, both the first triangle and the first The mode is selected from a second mode set including an exception mode for decoding the decoding target vertex using a third triangle different from the two triangles, and the decoding target vertex is decoded according to the mode.
  • the first triangle does not satisfy the condition, it may become possible to decode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to decode the vertices of the second triangle using a reference triangle suitable for decoding.
  • the decoding device of Example 16 may be the decoding device of Example 15, in which the third triangle is a triangle that shares a side with the first triangle.
  • the decoding device of Example 17 may be the decoding device of Example 15 or Example 16, in which the third triangle is a triangle that shares a side with the second triangle.
  • the decoding device of Example 18 may be the decoding device of any of Examples 15 to 17, wherein the third triangle is a triangle that shares an apex with the first triangle. .
  • the decoding device of Example 19 may be the decoding device of any of Examples 15 to 18, in which the third triangle is a triangle whose vertices have been decoded.
  • the decoding device of Example 20 may be the decoding device of any one of Examples 15 to 19, in which the third triangle is a triangle in which each side has connectivity. good.
  • the decoding device of Example 21 may be the decoding device of any of Examples 15 to 17, in which the third triangle is a triangle having a dummy apex.
  • the decoding device of Example 22 is the decoding device of any of Examples 15 to 21, wherein the exception mode includes a plurality of submodes, and the circuit further includes a mode in which the exception mode is selected as the mode.
  • the decoding device may select a submode for decoding the decoding target vertex from among the plurality of submodes.
  • the decoding device of Example 23 may be the decoding device of Example 22, and the circuit may be a decoding device that decodes a parameter indicating the submode.
  • the decoding device of Example 24 may be the decoding device of Example 22 or 23, in which the circuit determines the third triangle according to the submode.
  • the decoding device of Example 25 may be the decoding device of any of Examples 22 to 24, wherein the circuit determines the vertex of the third triangle according to the submode. .
  • the decoding device of Example 26 may be any of the decoding devices of Examples 15 to 25, in which the condition includes that each vertex of the first triangle has been decoded. good.
  • the decoding device of Example 27 is the decoding device of any of Examples 15 to 26, wherein the condition includes that each side of the first triangle has connectivity. There may be.
  • the decoding device of Example 28 is the decoding device of any one of Examples 15 to 27, in which, when the circuit decodes the decoding target vertex according to the exception mode, the decoding device uses the third triangle.
  • the decoding device may be a decoding device that decodes an angle indicating a target vertex.
  • This may make it possible to prevent the code amount from becoming too large. That is, it may be possible to suppress the worst case code amount.
  • the first triangle when the first triangle satisfies the condition, the first triangle is selected from among the first mode set including the normal mode for encoding the encoding target vertex of the second triangle using the first triangle.
  • the method includes the steps of: selecting the mode from a second mode set including an exception mode for encoding vertices; and encoding the vertices to be encoded according to the mode.
  • the first triangle does not satisfy the condition, it may be possible to encode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to encode the vertices of the second triangle using a reference triangle suitable for encoding.
  • the decoding target vertex when the first triangle satisfies the condition, is selected from a first mode set including a normal mode in which the decoding target vertex of the second triangle is decoded using the first triangle. and, if the first triangle does not satisfy the conditions, an exception mode of decoding the target vertex using a third triangle that is different from the first triangle and the second triangle.
  • the decoding method may include the steps of: selecting the mode from a second mode set including the second mode set; and decoding the decoding target vertex according to the mode.
  • the first triangle does not satisfy the condition, it may become possible to decode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to decode the vertices of the second triangle using a reference triangle suitable for decoding.
  • a three-dimensional mesh is a set of multiple surfaces, and represents, for example, a three-dimensional object. Furthermore, the three-dimensional mesh is mainly composed of vertex information, connection information, and attribute information. A three-dimensional mesh may be expressed as a polygon mesh or a mesh. Furthermore, the three-dimensional mesh may have temporal changes. The three-dimensional mesh may include metadata regarding vertex information, connection information, and attribute information, and may also include other additional information.
  • Vertex information is information indicating a vertex.
  • vertex information indicates the position of a vertex in three-dimensional space.
  • the vertices correspond to the vertices of the surfaces that constitute the three-dimensional mesh.
  • Vertex information may be expressed as "Geometry.” Additionally, vertex information may be expressed as position information.
  • Connection information is information indicating connections between vertices.
  • the connection information indicates connections for configuring faces or edges of a three-dimensional mesh.
  • Connection information is sometimes expressed as "Connectivity.”
  • connection information may also be expressed as surface information.
  • Attribute information is information indicating attributes of vertices or faces.
  • the attribute information indicates attributes such as a color, an image, and a normal vector associated with a vertex or a face. Attribute information is sometimes expressed as "Texture.”
  • a surface is an element that constitutes a three-dimensional mesh. Specifically, a surface is a polygon on a plane in three-dimensional space. For example, a surface may be defined as a triangle in three-dimensional space.
  • Plane A plane is a two-dimensional plane in three-dimensional space.
  • a polygon is formed on a plane, and a plurality of polygons are formed on a plurality of planes.
  • Bitstream corresponds to encoded information.
  • a bitstream may also be expressed as a stream, a coded bitstream, a compressed bitstream, or a coded signal.
  • Encoding and decoding may be replaced with expressions such as storing, including, writing, describing, signalling, transmitting, notifying, preserving, or compressing. These expressions may be interchanged.
  • encoding information may include including the information in a bitstream.
  • encoding information into a bitstream may mean encoding information to produce a bitstream that includes the encoded information.
  • decode may be replaced with expressions such as read, decipher, read, load, derive, obtain, receive, extract, restore, reconstruct, decompress, or decompress. Often these expressions may be interchanged.
  • decoding information may be obtaining information from a bitstream.
  • decoding information from a bitstream may mean decoding the bitstream to obtain information contained in the bitstream.
  • ordinal numbers such as first and second are sometimes given to constituent elements. These ordinal numbers may be changed as appropriate. Furthermore, ordinal numbers may be newly given or removed from the constituent elements. Additionally, these ordinal numbers may be attached to elements to identify them, and may not correspond to any meaningful order.
  • FIG. 1 is a conceptual diagram showing a three-dimensional mesh according to this embodiment.
  • a three-dimensional mesh is composed of multiple surfaces. For example, each face is a triangle. The vertices of these triangles are defined in three-dimensional space.
  • the three-dimensional mesh represents a three-dimensional object. Each side may have a color or an image.
  • FIG. 2 is a conceptual diagram showing the basic elements of the three-dimensional mesh according to this embodiment.
  • a three-dimensional mesh is composed of vertex information, connection information, and attribute information.
  • the vertex information indicates the position of the vertex of the surface in three-dimensional space.
  • Connection information indicates connections between vertices.
  • a surface can be identified by vertex information and connection information. In other words, a three-dimensional object without color is formed in a three-dimensional space using the vertex information and connection information.
  • Attribute information may be associated with vertices or surfaces. Attribute information associated with a vertex may be expressed as "Attribute Per Point.” The attribute information associated with a vertex may indicate the attribute of the vertex itself, or may indicate the attribute of a surface connected to the vertex.
  • a color may be associated with a vertex as attribute information.
  • the color associated with the vertex may be the color of the vertex or the color of the surface connected to the vertex.
  • the color of the surface may be the average of multiple colors associated with multiple vertices of the surface.
  • a normal vector may be associated with a vertex or a surface as attribute information. Such normal vectors can represent the front and back sides of a surface.
  • a two-dimensional image may be associated with the surface as attribute information.
  • a two-dimensional image associated with a surface is also expressed as a texture image or an "attribute map.”
  • information indicating mapping between a surface and a two-dimensional image may be associated with the surface as attribute information. Information indicating such mapping may be expressed as mapping information, vertex information of a texture image, or "Attribute UV Coordinate.”
  • information such as colors, images, and moving images used as attribute information may be expressed as "Parametric Space.”
  • texture can be reflected on the three-dimensional object. That is, a three-dimensional object having color is formed in a three-dimensional space using vertex information, connection information, and attribute information.
  • the attribute information is associated with a vertex or a face, but it may also be associated with an edge.
  • FIG. 3 is a conceptual diagram showing mapping according to this embodiment.
  • a region of a two-dimensional image in a two-dimensional plane can be mapped onto a surface of a three-dimensional mesh in three-dimensional space.
  • the coordinate information of the area in the two-dimensional image is associated with the surface of the three-dimensional mesh.
  • the image of the region mapped in the two-dimensional image is reflected on the surface of the three-dimensional mesh.
  • mapping the two-dimensional image used as attribute information can be separated from the three-dimensional mesh.
  • a two-dimensional image may be encoded using an image encoding method or a video encoding method.
  • FIG. 4 is a block diagram showing a configuration example of an encoding/decoding system according to this embodiment.
  • the encoding/decoding system includes an encoding device 100 and a decoding device 200.
  • the encoding device 100 obtains a three-dimensional mesh and encodes the three-dimensional mesh into a bitstream.
  • the encoding device 100 then outputs the bitstream to the network 300.
  • the bitstream includes an encoded three-dimensional mesh and control information for decoding the encoded three-dimensional mesh.
  • the information on the three-dimensional mesh is compressed.
  • the network 300 transmits the bitstream from the encoding device 100 to the decoding device 200.
  • the network 300 may be the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof.
  • the network 300 is not necessarily limited to two-way communication, but may be a one-way communication network for digital terrestrial broadcasting, satellite broadcasting, or the like.
  • the network 300 may be replaced by a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).
  • a recording medium such as a DVD (Digital Versatile Disc) or a BD (Blu-Ray Disc (registered trademark)).
  • the decoding device 200 obtains a bitstream and decodes a three-dimensional mesh from the bitstream. By decoding the three-dimensional mesh, the information of the three-dimensional mesh is expanded. For example, the decoding device 200 decodes the three-dimensional mesh according to a decoding method that corresponds to the encoding method used by the encoding device 100 to encode the three-dimensional mesh. That is, the encoding device 100 and the decoding device 200 perform encoding and decoding according to mutually corresponding encoding methods and decoding methods.
  • the three-dimensional mesh before encoding can also be expressed as the original three-dimensional mesh.
  • the three-dimensional mesh after decoding is also expressed as a reconstructed three-dimensional mesh.
  • FIG. 5 is a block diagram showing a configuration example of encoding device 100 according to this embodiment.
  • the encoding device 100 includes a vertex information encoder 101, a connection information encoder 102, and an attribute information encoder 103.
  • the vertex information encoder 101 is an electric circuit that encodes vertex information. For example, vertex information encoder 101 encodes vertex information into a bitstream according to a format defined for vertex information.
  • connection information encoder 102 is an electric circuit that encodes connection information.
  • the connection information encoder 102 encodes the connection information into a bitstream according to a format defined for the connection information.
  • the attribute information encoder 103 is an electric circuit that encodes attribute information. For example, the attribute information encoder 103 encodes the attribute information into a bitstream according to a format defined for the attribute information.
  • Variable length encoding or fixed length encoding may be used to encode the vertex information, connection information, and attribute information.
  • the variable length coding may correspond to Huffman coding, context adaptive binary arithmetic coding (CABAC), or the like.
  • the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be integrated. Alternatively, each of the vertex information encoder 101, the connection information encoder 102, and the attribute information encoder 103 may be further subdivided into a plurality of constituent elements.
  • FIG. 6 is a block diagram showing another configuration example of the encoding device 100 according to the present embodiment.
  • the encoding device 100 includes a preprocessor 104 and a postprocessor 105 in addition to the configuration shown in FIG.
  • the preprocessor 104 is an electric circuit that processes vertex information, connection information, and attribute information before encoding them.
  • the preprocessor 104 may perform conversion processing, separation processing, multiplexing processing, etc. on the three-dimensional mesh before encoding. More specifically, for example, the preprocessor 104 may separate vertex information, connection information, and attribute information from the three-dimensional mesh before encoding.
  • the post-processor 105 is an electric circuit that processes vertex information, connection information, and attribute information after encoding them.
  • the post-processor 105 may perform conversion processing, separation processing, multiplexing processing, etc. on the encoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 105 may multiplex the encoded vertex information, connection information, and attribute information into a bitstream.
  • the post-processor 105 may further perform variable length encoding on the encoded vertex information, connection information, and attribute information.
  • FIG. 7 is a block diagram showing a configuration example of decoding device 200 according to this embodiment.
  • the decoding device 200 includes a vertex information decoder 201, a connection information decoder 202, and an attribute information decoder 203.
  • the vertex information decoder 201 is an electric circuit that decodes vertex information. For example, vertex information decoder 201 decodes vertex information from a bitstream according to a format defined for vertex information.
  • connection information decoder 202 is an electric circuit that decodes connection information. For example, connection information decoder 202 decodes connection information from the bitstream according to a defined format for connection information.
  • the attribute information decoder 203 is an electric circuit that decodes attribute information. For example, the attribute information decoder 203 decodes the attribute information from the bitstream according to a format defined for the attribute information.
  • Variable length decoding or fixed length decoding may be used to decode the vertex information, connection information, and attribute information.
  • Variable length decoding may correspond to Huffman coding, context adaptive binary arithmetic coding (CABAC), or the like.
  • the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be integrated. Alternatively, each of the vertex information decoder 201, the connection information decoder 202, and the attribute information decoder 203 may be further subdivided into a plurality of constituent elements.
  • FIG. 8 is a block diagram showing another configuration example of the decoding device 200 according to the present embodiment.
  • the decoding device 200 includes a preprocessor 204 and a postprocessor 205 in addition to the configuration shown in FIG.
  • the preprocessor 204 is an electric circuit that processes vertex information, connection information, and attribute information before decoding them.
  • the preprocessor 204 may perform conversion processing, separation processing, multiplexing processing, etc. on the bitstream before decoding the vertex information, connection information, and attribute information.
  • the preprocessor 204 separates a sub-bitstream corresponding to vertex information, a sub-bitstream corresponding to connection information, and a sub-bitstream corresponding to attribute information from the bitstream. Good too. Further, for example, the preprocessor 204 may perform variable length decoding on the bitstream in advance before decoding the vertex information, connection information, and attribute information.
  • the post-processor 205 is an electric circuit that processes the vertex information, connection information, and attribute information after decoding them.
  • the post-processor 205 may perform conversion processing, separation processing, multiplexing processing, etc. on the decoded vertex information, connection information, and attribute information. More specifically, for example, the post-processor 205 may multiplex the decoded vertex information, connection information, and attribute information into a three-dimensional mesh.
  • Vertex information connection information and attribute information are encoded and stored in the bitstream.
  • connection information and attribute information are encoded and stored in the bitstream. The relationship between this information and the bitstream is shown below.
  • FIG. 9 is a conceptual diagram showing an example of the configuration of a bitstream according to this embodiment.
  • vertex information, connection information, and attribute information are combined in the bitstream.
  • vertex information, connection information, and attribute information may be included in one file.
  • FIG. 10 is a conceptual diagram showing another configuration example of the bitstream according to the present embodiment.
  • multiple files are included in the bitstream, and vertex information, connection information, and attribute information are stored in different files.
  • a file containing vertex information, a file containing connection information, and a file containing attribute information are shown here, the storage format is not limited to such an example.
  • two types of information among vertex information, connection information, and attribute information may be included in one file, and the remaining one type of information may be included in another file.
  • this information may be divided and stored in more files.
  • multiple parts of vertex information may be stored in multiple files
  • multiple parts of connection information may be stored in multiple files
  • multiple parts of attribute information may be stored in multiple files. may be done. These multiple parts may correspond to multiple temporally different parts, spatially different parts, or different surfaces. .
  • FIG. 11 is a conceptual diagram showing another configuration example of the bitstream according to the present embodiment.
  • a bitstream is composed of a plurality of separable sub-bitstreams, and vertex information, connection information, and attribute information are stored in different sub-bitstreams.
  • the storage format is not limited to this example.
  • two types of information among vertex information, connection information, and attribute information may be included in one sub-bitstream, and the remaining one type of information may be included in another sub-bitstream.
  • attribute information such as a two-dimensional image may be stored in a sub-bitstream compliant with an image encoding method, separately from sub-bitstreams of vertex information and connection information.
  • FIG. 12 is a block diagram showing a specific example of the encoding/decoding system according to this embodiment.
  • the encoding/decoding system includes a three-dimensional data encoding system 110, a three-dimensional data decoding system 210, and an external connector 310.
  • the three-dimensional data encoding system 110 includes a controller 111, an input/output processor 112, a three-dimensional data encoder 113, a three-dimensional data generator 115, and a system multiplexer 114.
  • the three-dimensional data decoding system 210 includes a controller 211 , an input/output processor 212 , a three-dimensional data decoder 213 , a system demultiplexer 214 , a presenter 215 , and a user interface 216 .
  • sensor data is input from the sensor terminal to the three-dimensional data generator 115.
  • the three-dimensional data generator 115 generates three-dimensional data such as point group data or mesh data from the sensor data, and inputs it to the three-dimensional data encoder 113 .
  • the three-dimensional data generator 115 generates vertex information, and generates connection information and attribute information corresponding to the vertex information.
  • the three-dimensional data generator 115 may process vertex information when generating connection information and attribute information. For example, the three-dimensional data generator 115 may reduce the amount of data by deleting duplicate vertices, or transform vertex information (position shift, rotation, normalization, etc.).
  • the three-dimensional data generator 115 may also render attribute information.
  • the three-dimensional data generator 115 is a component of the three-dimensional data encoding system 110 in FIG. 12, it may be arranged externally and independently from the three-dimensional data encoding system 110.
  • a sensor terminal that provides sensor data for generating three-dimensional data may be, for example, a moving object such as a car, a flying object such as an airplane, a mobile terminal, a camera, or the like. Further, a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor, or a range finder, a stereo camera, or a combination of a plurality of monocular cameras may be used as the sensor terminal.
  • a distance sensor such as a LIDAR, a millimeter wave radar, an infrared sensor, or a range finder, a stereo camera, or a combination of a plurality of monocular cameras may be used as the sensor terminal.
  • Sensor data includes object distance (position), monocular camera image, stereo camera image, color, reflectance, sensor attitude, direction, gyro, sensing position (GPS information or altitude), speed, acceleration, sensing time, and temperature. , atmospheric pressure, humidity, or magnetism.
  • the three-dimensional data encoder 113 corresponds to the encoding device 100 shown in FIG. 5 etc.
  • the three-dimensional data encoder 113 encodes three-dimensional data and generates encoded data.
  • the three-dimensional data encoder 113 generates control information when encoding three-dimensional data.
  • the three-dimensional data encoder 113 then inputs the encoded data together with the control information to the system multiplexer 114.
  • the encoding method for three-dimensional data may be an encoding method using geometry or an encoding method using a video codec.
  • the encoding method using geometry can also be expressed as a geometry-based encoding method.
  • An encoding method using a video codec is also expressed as a video-based encoding method.
  • the system multiplexer 114 multiplexes the encoded data and control information input from the three-dimensional data encoder 113, and generates multiplexed data using a prescribed multiplexing method.
  • the system multiplexer 114 may multiplex other media such as video, audio, subtitles, application data, or document files, reference time information, etc., along with encoded data of three-dimensional data and control information. Additionally, system multiplexer 114 may multiplex attribute information associated with sensor data or three-dimensional data.
  • the multiplexed data has a file format for storage, a packet format for transmission, etc.
  • ISOBMFF or an ISOBMFF-based method may be used.
  • MPEG-DASH, MMT, MPEG-2 TS Systems, RTP, or the like may be used.
  • the multiplexed data is then output as a transmission signal to the external connector 310 by the input/output processor 112.
  • the multiplexed data may be transmitted as a transmission signal by wire or wirelessly.
  • the multiplexed data is stored in internal memory or storage.
  • the multiplexed data may be transmitted to a cloud server via the Internet or may be stored in an external storage device.
  • multiplexed data is transmitted or stored in a manner that depends on the medium for transmission or storage, such as broadcasting or communication.
  • the communication protocol http, ftp, TCP, UDP, IP, or a combination thereof may be used.
  • a PULL type communication method or a PUSH type communication method may be used.
  • Ethernet registered trademark
  • USB registered trademark
  • RS-232C HDMI (registered trademark)
  • coaxial cable etc.
  • 3G/4G/5G defined by 3GPP registered trademark
  • IEEE wireless LAN
  • Wi-Fi Wi-Fi
  • Bluetooth or millimeter waves
  • DVB-T2 DVB-S2, DVB-C2, ATSC3.0, or ISDB-S3
  • the sensor data may be input to the three-dimensional data generator 115 or the system multiplexer 114.
  • the three-dimensional data or encoded data may be output as a transmission signal to the external connector 310 via the input/output processor 112 as is.
  • the transmission signal output from the three-dimensional data encoding system 110 is input to the three-dimensional data decoding system 210 via the external connector 310.
  • each operation of the three-dimensional data encoding system 110 may be controlled by a controller 111 that executes an application program.
  • a transmission signal is input to an input/output processor 212.
  • the input/output processor 212 decodes multiplexed data having a file format or packet format from the transmission signal and inputs the multiplexed data to the system demultiplexer 214 .
  • the system demultiplexer 214 acquires encoded data and control information from the multiplexed data and inputs them to the three-dimensional data decoder 213.
  • the system demultiplexer 214 may extract other media or reference time information, etc. from the multiplexed data.
  • the three-dimensional data decoder 213 corresponds to the decoding device 200 shown in FIG. 7 etc.
  • the three-dimensional data decoder 213 decodes three-dimensional data from encoded data based on a predefined encoding method.
  • the three-dimensional data is then presented to the user by the presenter 215.
  • additional information such as sensor data may be input to the presenter 215.
  • the presenter 215 may present three-dimensional data based on additional information.
  • user instructions may be input from the user terminal to the user interface 216.
  • the presenter 215 may then present three-dimensional data based on the input instructions.
  • the input/output processor 212 may acquire three-dimensional data and encoded data from the external connector 310.
  • each operation of the three-dimensional data decoding system 210 may be controlled by a controller 211 that executes an application program.
  • FIG. 13 is a conceptual diagram showing a configuration example of point cloud data according to this embodiment.
  • Point cloud data is data of a point cloud indicating a three-dimensional object.
  • a point cloud is composed of a plurality of points, and has position information indicating the three-dimensional coordinate position (Position) of each point, and attribute information indicating the attribute (Attribute) of each point. .
  • Position information is also expressed as geometry.
  • the type of attribute information may be, for example, color or reflectance.
  • One point may be associated with attribute information regarding one type, one point may be associated with attribute information regarding multiple different types, or one point may be associated with attribute information regarding the same type. may be associated with attribute information having multiple values.
  • FIG. 14 is a conceptual diagram showing an example of a data file of point cloud data according to this embodiment.
  • the position information is information indicating a three-dimensional coordinate position on three axes of x, y, and z
  • the attribute information is information indicating a color in RGB.
  • a PLY file or the like may be used as a typical data file of point cloud data.
  • FIG. 15 is a conceptual diagram showing a configuration example of mesh data according to this embodiment.
  • Mesh data is data used for CG (Computer Graphics) and the like, and is three-dimensional mesh data that shows the three-dimensional shape of an object in a plurality of planes. Each surface is also expressed as a polygon and has a polygonal shape such as a triangle or a quadrilateral.
  • a three-dimensional mesh is composed of a plurality of edges and a plurality of faces in addition to a plurality of points forming a point group.
  • Each point is also expressed as a vertex or a position.
  • Each edge corresponds to a line segment connected by two vertices.
  • Each surface corresponds to an area surrounded by three or more sides.
  • the three-dimensional mesh has position information indicating the three-dimensional coordinate position of the vertex. Position information is also expressed as vertex information or geometry. Furthermore, the three-dimensional mesh has connection information indicating the relationship between a plurality of vertices forming an edge or a face. Connection information is also expressed as connectivity. Furthermore, the three-dimensional mesh has attribute information indicating attributes for vertices, edges, or faces. Attribute information in a three-dimensional mesh is also expressed as texture.
  • the attribute information may indicate color, reflectance, or normal vector for a vertex, edge, or surface.
  • the direction of the normal vector can represent the front and back sides of the surface.
  • An object file or the like may be used as the data file format of the mesh data.
  • FIG. 16 is a conceptual diagram showing an example of a data file of mesh data according to this embodiment.
  • the data file contains position information G(1) to G(N) of N vertices that make up the three-dimensional mesh, and attribute information A1(1) to A1(N) of N vertices. is included.
  • M pieces of attribute information A2(1) to A2(M) are included. Items of attribute information do not have to correspond one-to-one to vertices, and do not have to correspond one-to-one to faces. Further, attribute information may not exist.
  • connection information is indicated by a combination of vertex indexes.
  • attribute information may be written in a separate file.
  • a pointer to the content may be associated with a vertex, a surface, or the like.
  • attribute information indicating an image for a surface may be stored in a two-dimensional attribute map file. Then, the file name of the attribute map and the two-dimensional coordinate values in the attribute map may be written in the attribute information A2(1) to A2(M).
  • the method of specifying attribute information for a surface is not limited to these methods, and any method may be used.
  • FIG. 17 is a conceptual diagram showing the types of three-dimensional data according to this embodiment.
  • Point cloud data and mesh data may represent static or dynamic objects.
  • Static objects are objects that do not change over time, and dynamic objects are objects that change over time.
  • Static objects may correspond to three-dimensional data for any point in time.
  • point cloud data for an arbitrary point in time may be expressed as a PCC frame.
  • mesh data for an arbitrary point in time may be expressed as a mesh frame.
  • a PCC frame and a mesh frame may be simply expressed as a frame.
  • the area of the object may be limited to a certain range like normal video data, or may not be limited like map data.
  • the density of points or planes can be determined in various ways. Sparse point cloud data or sparse mesh data may be used, or dense point cloud data or dense mesh data may be used.
  • the apparatus, process, or syntax for encoding and decoding vertex information of a three-dimensional mesh in the present disclosure may be applied to encoding and decoding point clouds.
  • the apparatus, process, or syntax for encoding and decoding a point cloud in this disclosure may be applied to encoding and decoding vertex information of a three-dimensional mesh.
  • the device, process, or syntax for encoding and decoding attribute information of a point cloud in the present disclosure may be applied to encoding and decoding connection information or attribute information of a three-dimensional mesh. Further, the device, process, or syntax for encoding and decoding three-dimensional mesh connection information or attribute information in the present disclosure may be applied to encoding and decoding point cloud attribute information.
  • processing may be shared between encoding and decoding of point cloud data and encoding and decoding of mesh data. This can reduce the scale of circuits and software programs.
  • FIG. 18 is a block diagram showing a configuration example of the three-dimensional data encoder 113 according to this embodiment.
  • the three-dimensional data encoder 113 includes a vertex information encoder 121, an attribute information encoder 122, a metadata encoder 123, and a multiplexer 124.
  • the vertex information encoder 121, attribute information encoder 122, and multiplexer 124 may correspond to the vertex information encoder 101, attribute information encoder 103, post-processor 105, etc. in FIG.
  • the three-dimensional data encoder 113 encodes the three-dimensional data according to the geometry-based encoding method.
  • three-dimensional structures are taken into account.
  • attribute information is encoded using configuration information obtained in encoding vertex information.
  • vertex information, attribute information, and metadata included in three-dimensional data generated from sensor data are sent to a vertex information encoder 121, an attribute information encoder 122, and a metadata encoder 123.
  • the connection information included in the three-dimensional data may be treated in the same way as attribute information.
  • position information may be treated as vertex information.
  • the vertex information encoder 121 encodes the vertex information into compressed vertex information and outputs the compressed vertex information to the multiplexer 124 as encoded data. Additionally, the vertex information encoder 121 generates metadata of compressed vertex information and outputs it to the multiplexer 124 . Additionally, the vertex information encoder 121 generates configuration information and outputs it to the attribute information encoder 122.
  • the attribute information encoder 122 encodes the attribute information into compressed attribute information using the configuration information generated by the vertex information encoder 121, and outputs the compressed attribute information as encoded data to the multiplexer 124. Further, the attribute information encoder 122 generates metadata of compressed attribute information and outputs it to the multiplexer 124 .
  • the metadata encoder 123 encodes compressed metadata into compressed metadata, and outputs the compressed metadata to the multiplexer 124 as encoded data.
  • the metadata encoded by the metadata encoder 123 may be used for encoding vertex information and attribute information.
  • the multiplexer 124 multiplexes compressed vertex information, metadata of compressed vertex information, compressed attribute information, metadata of compressed attribute information, and compressed metadata into a bitstream.
  • the multiplexer 124 then inputs the bitstream to the system layer.
  • FIG. 19 is a block diagram showing a configuration example of the three-dimensional data decoder 213 according to the present embodiment.
  • the three-dimensional data decoder 213 includes a vertex information decoder 221, an attribute information decoder 222, a metadata decoder 223, and a demultiplexer 224.
  • the vertex information decoder 221, attribute information decoder 222, and demultiplexer 224 may correspond to the vertex information decoder 201, attribute information decoder 203, preprocessor 204, etc. in FIG. 8.
  • the three-dimensional data decoder 213 decodes the three-dimensional data according to the geometry-based encoding method.
  • three-dimensional structures are taken into account.
  • attribute information is decoded using configuration information obtained in decoding vertex information.
  • a bitstream is input to the demultiplexer 224 from the system layer.
  • the demultiplexer 224 separates compressed vertex information, compressed vertex information metadata, compressed attribute information, compressed attribute information metadata, and compressed metadata from the bitstream.
  • the compressed vertex information and the metadata of the compressed vertex information are input to the vertex information decoder 221.
  • the compressed attribute information and the metadata of the compressed attribute information are input to the attribute information decoder 222.
  • Metadata is input to metadata decoder 223.
  • the vertex information decoder 221 decodes the vertex information from the compressed vertex information using the metadata of the compressed vertex information. Additionally, the vertex information decoder 221 generates configuration information and outputs it to the attribute information decoder 222.
  • the attribute information decoder 222 decodes the attribute information from the compressed attribute information using the configuration information generated by the vertex information decoder 221 and the metadata of the compressed attribute information.
  • Metadata decoder 223 decodes metadata from compressed metadata. The metadata decoded by the metadata decoder 223 may be used for decoding vertex information and attribute information.
  • this metadata is metadata of vertex information and attribute information, and can be used in an application program.
  • FIG. 20 is a block diagram showing another configuration example of the three-dimensional data encoder 113 according to this embodiment.
  • the three-dimensional data encoder 113 includes a vertex image generator 131, an attribute image generator 132, a metadata generator 133, a video encoder 134, a metadata encoder 123, and a multiplexer 124.
  • the vertex image generator 131, attribute image generator 132, and video encoder 134 may correspond to the vertex information encoder 101, attribute information encoder 103, etc. in FIG. 6.
  • the three-dimensional data encoder 113 encodes the three-dimensional data according to the video-based encoding method.
  • a plurality of two-dimensional images are generated from three-dimensional data, and the plurality of two-dimensional images are encoded according to the video encoding method.
  • the video encoding method may be HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), or the like.
  • vertex information and attribute information included in three-dimensional data generated from sensor data are input to the metadata generator 133. Further, vertex information and attribute information are input to a vertex image generator 131 and an attribute image generator 132, respectively. Further, metadata included in the three-dimensional data is input to the metadata encoder 123.
  • the connection information included in the three-dimensional data may be treated in the same way as attribute information. Furthermore, in the case of point cloud data, position information may be treated as vertex information.
  • the metadata generator 133 generates map information of a plurality of two-dimensional images from vertex information and attribute information. The metadata generator 133 then inputs the map information to the vertex image generator 131, attribute image generator 132, and metadata encoder 123.
  • the vertex image generator 131 generates vertex images based on the vertex information and map information, and inputs the generated vertex images to the video encoder 134.
  • Attribute image generator 132 generates an attribute image based on the attribute information and map information and inputs it to video encoder 134 .
  • the video encoder 134 encodes the vertex image and the attribute image into compressed vertex information and compressed attribute information, respectively, according to the video encoding method, and outputs the compressed vertex information and compressed attribute information to the multiplexer 124 as encoded data. do. Further, the video encoder 134 generates metadata of compressed vertex information and metadata of compressed attribute information and outputs them to the multiplexer 124.
  • the metadata encoder 123 encodes compressed metadata into compressed metadata, and outputs the compressed metadata to the multiplexer 124 as encoded data.
  • Compressible metadata includes map information. Further, the metadata encoded by the metadata encoder 123 may be used for encoding vertex information and attribute information.
  • the multiplexer 124 multiplexes compressed vertex information, metadata of compressed vertex information, compressed attribute information, metadata of compressed attribute information, and compressed metadata into a bitstream.
  • the multiplexer 124 then inputs the bitstream to the system layer.
  • FIG. 21 is a block diagram showing another configuration example of the three-dimensional data decoder 213 according to this embodiment.
  • the three-dimensional data decoder 213 includes a vertex information generator 231, an attribute information generator 232, a video decoder 234, a metadata decoder 223, and a demultiplexer 224.
  • the vertex information generator 231, attribute information generator 232, and video decoder 234 may correspond to the vertex information decoder 201, attribute information decoder 203, etc. in FIG. 8.
  • the three-dimensional data decoder 213 decodes the three-dimensional data according to the video-based encoding method.
  • decoding according to the video-based encoding method a plurality of two-dimensional images are decoded according to the video encoding method, and three-dimensional data is generated from the plurality of two-dimensional images.
  • the video encoding method may be HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), or the like.
  • the metadata decoder 223 decodes metadata from compressed metadata.
  • the metadata decoded by the metadata decoder 223 includes map information used to generate vertex information and attribute information. Further, the metadata decoded by the metadata decoder 223 may be used for decoding vertex images and attribute images.
  • the vertex information encoder 144, connection information encoder 145, and texture encoder 143 correspond to the vertex information encoder 101, connection information encoder 102, attribute information encoder 103, etc. in FIG. Good too.
  • the two-dimensional data encoder 141 operates as a texture encoder 143, and encodes a texture corresponding to attribute information as two-dimensional data according to an image encoding method or a video encoding method, thereby creating a texture file. generate.
  • bitstream including a texture file, a mesh file, and a description file is generated.
  • These files may be multiplexed into a bitstream in a file format such as glTF (Graphics Language Transmission Format) or USD (Universal Scene Description).
  • glTF Graphics Language Transmission Format
  • USD Universal Scene Description
  • the three-dimensional data encoder 113 may include two mesh data encoders as the mesh data encoder 142.
  • one mesh data encoder encodes vertex information and connection information of a static 3D mesh
  • the other mesh data encoder encodes vertex information and connection information of a dynamic 3D mesh.
  • two mesh files may be included in the bitstream.
  • one mesh file corresponds to a static three-dimensional mesh
  • the other mesh file corresponds to a dynamic three-dimensional mesh.
  • the static 3D mesh may be an intra-frame 3D mesh encoded using intra prediction
  • the dynamic 3D mesh may be an interframe 3D mesh encoded using inter prediction.
  • the original mesh may be used.
  • difference information between the vertex information or connection information of the three-dimensional mesh of the intra-frame and the vertex information or connection information of the three-dimensional mesh of the inter-frame may be used.
  • FIG. 23 is a conceptual diagram showing a specific example of decoding processing according to this embodiment.
  • FIG. 23 shows a three-dimensional data decoder 213, a description decoder 248, and a presenter 247.
  • the three-dimensional data decoder 213 includes a two-dimensional data decoder 241, a mesh data decoder 242, and a mesh reconstructor 246.
  • the two-dimensional data decoder 241 includes a texture decoder 243.
  • the mesh data decoder 242 includes a vertex information decoder 244 and a connection information decoder 245.
  • the two-dimensional data decoder 241 operates as a texture decoder 243, and decodes a texture corresponding to attribute information from a texture file as two-dimensional data according to an image encoding method or a video encoding method.
  • the mesh data decoder 242 operates as a vertex information decoder 244 and a connection information decoder 245, and decodes vertex information and connection information from the mesh file.
  • Mesh data decoder 242 may further decode mapping information for textures from the mesh file.
  • description decoder 248 decodes a description corresponding to metadata such as text data from the description file.
  • Description decoder 248 may decode the description at the system layer.
  • description decoder 248 may be included in system demultiplexer 214 of FIG. 12.
  • the mesh reconstructor 246 reconstructs a three-dimensional mesh from the vertex information, connection information, and texture according to the description.
  • the presenter 247 renders and outputs a three-dimensional mesh according to the description.
  • a three-dimensional mesh is reconstructed and output from the bitstream including the texture file, mesh file, and description file.
  • the three-dimensional data decoder 213 may include two mesh data decoders as the mesh data decoder 242.
  • one mesh data decoder decodes vertex information and connection information of a static three-dimensional mesh
  • the other mesh data decoder decodes vertex information and connection information of a dynamic three-dimensional mesh.
  • two mesh files may be included in the bitstream.
  • one mesh file corresponds to a static three-dimensional mesh
  • the other mesh file corresponds to a dynamic three-dimensional mesh.
  • the static 3D mesh may be an intra-frame 3D mesh encoded using intra prediction
  • the dynamic 3D mesh may be an interframe 3D mesh encoded using inter prediction.
  • the original mesh may be used.
  • difference information between the vertex information or connection information of the three-dimensional mesh of the intra-frame and the vertex information or connection information of the three-dimensional mesh of the inter-frame may be used.
  • the dynamic three-dimensional mesh encoding method is sometimes called DMC (Dynamic Mesh Coding). Further, a dynamic three-dimensional mesh video-based coding method is sometimes referred to as V-DMC (Video-based Dynamic Mesh Coding).
  • DMC Dynamic Mesh Coding
  • V-DMC Video-based Dynamic Mesh Coding
  • the point cloud encoding method is sometimes called PCC (Point Cloud Compression). Further, the video-based encoding method for point clouds is sometimes referred to as V-PCC (Video-based Point Cloud Compression). Further, the geometry-based encoding method for point clouds is sometimes called G-PCC (Geometry-based Point Cloud Compression).
  • FIG. 24 is a block diagram showing an implementation example of encoding device 100 according to this embodiment.
  • Encoding device 100 includes a circuit 151 and memory 152.
  • a plurality of components of the encoding device 100 shown in FIG. 5 etc. are implemented by the circuit 151 and memory 152 shown in FIG. 24.
  • the circuit 151 is a circuit that performs information processing and is a circuit that can access the memory 152.
  • circuit 151 is a dedicated or general-purpose electrical circuit that encodes a three-dimensional mesh.
  • Circuit 151 may be a processor such as a CPU. Further, the circuit 151 may be a collection of a plurality of electric circuits.
  • the memory 152 is a dedicated or general-purpose memory in which information for the circuit 151 to encode the three-dimensional mesh is stored.
  • Memory 152 may be an electrical circuit or may be connected to circuit 151. Further, the memory 152 may be included in the circuit 151. Further, the memory 152 may be a collection of a plurality of electric circuits. Further, the memory 152 may be a magnetic disk, an optical disk, or the like, or may be expressed as a storage, a recording medium, or the like. Further, the memory 152 may be a nonvolatile memory or a volatile memory.
  • the memory 152 may store a three-dimensional mesh or a bitstream. Further, the memory 152 may store a program for the circuit 151 to encode a three-dimensional mesh.
  • the encoding device 100 not all of the plurality of components shown in FIG. 5 etc. need to be implemented, and all of the plurality of processes shown here need not be performed. Some of the multiple components shown in FIG. 5 etc. may be included in other devices, and some of the multiple processes shown here may be executed by other devices. . Furthermore, in the encoding device 100, a plurality of components of the present disclosure may be implemented in any combination, or a plurality of processes of the present disclosure may be performed in any combination.
  • FIG. 25 is a block diagram showing an implementation example of the decoding device 200 according to this embodiment.
  • the decoding device 200 includes a circuit 251 and a memory 252.
  • a plurality of components of the decoding device 200 shown in FIG. 7 etc. are implemented by the circuit 251 and memory 252 shown in FIG. 25.
  • the circuit 251 is a circuit that performs information processing and is a circuit that can access the memory 252.
  • circuit 251 is a dedicated or general-purpose electrical circuit that decodes a three-dimensional mesh.
  • Circuit 251 may be a processor such as a CPU. Further, the circuit 251 may be a collection of a plurality of electric circuits.
  • the memory 252 is a dedicated or general-purpose memory in which information for the circuit 251 to decode the three-dimensional mesh is stored.
  • Memory 252 may be an electrical circuit or may be connected to circuit 251. Further, the memory 252 may be included in the circuit 251. Furthermore, the memory 252 may be a collection of a plurality of electric circuits. Further, the memory 252 may be a magnetic disk, an optical disk, or the like, or may be expressed as a storage, a recording medium, or the like. Further, the memory 252 may be a nonvolatile memory or a volatile memory.
  • the memory 252 may store a three-dimensional mesh or a bitstream. Further, the memory 252 may store a program for the circuit 251 to decode the three-dimensional mesh.
  • the decoding device 200 not all of the multiple components shown in FIG. 7 etc. need to be implemented, and all of the multiple processes shown here need not be performed. Some of the multiple components shown in FIG. 7 etc. may be included in other devices, and some of the multiple processes shown here may be executed by other devices. . Furthermore, in the decoding device 200, a plurality of components of the present disclosure may be implemented in any combination, and a plurality of processes of the present disclosure may be performed in any combination.
  • the encoding method and decoding method including the steps performed by each component of the encoding device 100 and decoding device 200 of the present disclosure may be executed by any device or system.
  • part or all of the encoding method and decoding method may be executed by a computer including a processor, memory, input/output circuit, and the like.
  • the encoding method and the decoding method may be executed by the computer executing a program for causing the computer to execute the encoding method and the decoding method.
  • a program or a bitstream may be recorded on a non-temporary computer-readable recording medium such as a CD-ROM.
  • An example of the program may be a bitstream.
  • a bitstream including an encoded three-dimensional mesh includes a syntax element for causing the decoding device 200 to decode the three-dimensional mesh. Then, the bitstream causes the decoding device 200 to decode the three-dimensional mesh according to the syntax elements included in the bitstream. Therefore, a bitstream can play a role similar to a program.
  • the above bitstream may be an encoded bitstream that includes an encoded three-dimensional mesh, or a multiplexed bitstream that includes an encoded three-dimensional mesh and other information.
  • each component of the encoding device 100 and the decoding device 200 may be configured with dedicated hardware, general-purpose hardware that executes the above program, etc., or a combination thereof. It may be composed of.
  • the general-purpose hardware may include a memory in which a program is recorded, a general-purpose processor that reads the program from the memory, and executes the program.
  • the memory may be a semiconductor memory or a hard disk, and the general-purpose processor may be a CPU or the like.
  • the dedicated hardware may be composed of a memory, a dedicated processor, and the like.
  • a dedicated processor may execute the encoding method and decoding method with reference to a memory for recording data.
  • each component of the encoding device 100 and the decoding device 200 may be an electric circuit, as described above. These electric circuits may constitute one electric circuit as a whole, or may be separate electric circuits. Furthermore, these electric circuits may correspond to dedicated hardware or may correspond to general-purpose hardware that executes the above programs and the like. Further, encoding device 100 and decoding device 200 may be implemented as an integrated circuit.
  • the encoding device 100 may be a transmitting device that transmits a three-dimensional mesh.
  • Decoding device 200 may be a receiving device that receives a three-dimensional mesh.
  • ⁇ Reference example regarding encoding and decoding of vertex information> For example, in encoding and decoding vertex information, a parallelogram prediction method for predicting the positions of vertices may be used.
  • FIG. 26 is a conceptual diagram showing a parallelogram prediction method according to a reference example.
  • the vertex information of the first vertex A, the second vertex B, and the third vertex C of the first triangle is encoded
  • the vertex information of the second triangle having the same second vertex B and third vertex C as the first triangle is encoded.
  • Vertex information of the fourth vertex D is encoded.
  • the position of the fourth vertex D is predicted from the first vertex A, the second vertex B, and the third vertex C using the parallelogram prediction method, and errors in the prediction are encoded.
  • the vertex P of a virtual parallelogram ABCP having a diagonal line BC and a diagonal line AP is calculated as a predicted point.
  • an error vector indicating the difference between vertex D and vertex P in the three-dimensional space is encoded.
  • a more complex version uses multiple triangles. This approach slows down the compression process but provides better predictions.
  • a polynomial prediction method for predicting the position of a vertex may be used.
  • polynomial prediction methods local configurations are used to predict the positions of new vertices. Specifically, several vertex coordinate values within a local neighborhood are fed to a polynomial function with predetermined coefficients, and the position of a new vertex is predicted according to the polynomial function. The vertices used for predicting new vertices are selected according to certain conditions.
  • FIG. 27 is a conceptual diagram showing a polynomial prediction method according to a reference example.
  • the predicted point of the next vertex is derived using the four encoded vertices. That is, in FIG. 27, the coordinate values of vertex #1, vertex #2, vertex #3, and vertex #4 are used to predict vertex #5. Similar to the parallelogram prediction method, an error vector indicating the error to the prediction is encoded.
  • vertex information is encoded using an error vector calculated according to a prediction method such as a parallelogram prediction method or a polynomial prediction method. Regardless of the complexity of these prediction schemes, it is difficult to ensure that the magnitude of these errors is within a certain range. For example, obtuse triangles tend to have larger error vectors than equilateral triangles. Furthermore, the smoothness of the shape and the size of the surface affect the magnitude of the error. Therefore, more bits may be used for signaling vertex information.
  • the non-common vertices of adjacent triangles with respect to the reference triangle are encoded using three angles: a dihedral angle and two other angles. That is, when vertex information is encoded using the spatial relationships between adjacent triangles, only three angles are encoded. This may reduce the number of bits.
  • the sign of the angle may be used to indicate the orientation of the angle with respect to the side.
  • a predetermined rule may be adopted in which positive values indicate a clockwise direction with respect to the reference and negative values indicate a counterclockwise direction with respect to the reference.
  • the two-dimensional plane of the second triangle can be found by the dihedral angle formed between the first triangle, which is the reference triangle, and the second triangle, which is the processing target triangle.
  • the complexity of three dimensions can be reduced to two dimensions where simpler plane laws can be applied.
  • the dihedral angle between adjacent surfaces may be calculated using surface normals. Then, a corresponding triangle that is a one-to-one virtual congruent triangle with respect to the second triangle is constructed on the plane of the first triangle. The first triangle and the corresponding triangle have a common side and are located on opposite sides of the common side. The vertices of the corresponding triangles are identified using two angles. Therefore, the apex of the second triangle is determined by the dihedral angle formed by the first triangle and the second triangle, and the two angles for specifying the apex of the corresponding triangle that is congruent with the second triangle. is specified.
  • the values of these angles tend to follow a certain distribution. Therefore, it is also possible to improve the compression ratio by utilizing such a tendency.
  • the distribution of angles may be examined and the difference with respect to the mean of the distribution may be encoded.
  • FIG. 28 is a flowchart illustrating an example of encoding processing according to this aspect.
  • a first vertex, a second vertex, and a third vertex are derived from the three-dimensional mesh (S101).
  • the first vertex, the second vertex, and the third vertex form a first triangle on the first plane of the three-dimensional mesh.
  • the first vertex, the second vertex, and the third vertex are three encoded vertices.
  • the first angle, second angle, and third angle are derived to derive the fourth vertex of the second triangle formed on the second plane of the three-dimensional mesh using the common side with the first triangle. (S102).
  • the first angle, the second angle, and the third angle are encoded into a bitstream (S103).
  • FIG. 29 is a flowchart showing a specific example of encoding processing according to this aspect.
  • each vertex coordinate value of the first surface is encoded from the three-dimensional mesh (S111).
  • the first face may be randomly selected.
  • the processing mode of the vertex is specified according to the written traversal symbol.
  • Vertices are encoded using coordinate values or angles. For example, if the traversal symbol indicates coordinates (coordinates in S114), the vertex coordinate values (x, y, z) are directly encoded (S115). This method may be chosen at the beginning of encoding, when no vertices have yet been encoded. Further, for example, if the traversal symbol indicates an angle (angle in S114), the vertex is encoded using three angles (S116).
  • Additional residuals may be calculated and signaled separately after the angle signaling.
  • the fractional part of the angular floating-point value is encoded as a residual value. It's okay. The traversal process continues until all three-dimensional meshes are traversed (S119).
  • a traversal symbol indicating any of skip, end, and execution may be encoded.
  • Skip corresponds to the vertex being already processed.
  • Termination corresponds to all vertices being processed.
  • the execution corresponds to the vertex to be processed.
  • a traversal symbol indicating execution is encoded, a processing mode indicating either coordinates or angle may be encoded separately from the traversal symbol. Processing mode may also be expressed as mode, operation mode, prediction mode, encoding mode, or decoding mode.
  • FIG. 30 is a flowchart showing another specific example of the encoding process according to this aspect.
  • the encoding process of the reference example is integrated into the encoding process of FIG.
  • each vertex coordinate value of the first surface is encoded from the three-dimensional mesh (S111).
  • the processing mode of the vertex is specified according to the written traversal symbol.
  • Vertices are encoded using coordinates, angles, parallelograms or polynomials. For example, if the traversal symbol indicates coordinates (coordinates in S114), the vertex coordinate values (x, y, z) are directly encoded (S115). This method may be chosen at the beginning of encoding, when no vertices have yet been encoded. Further, for example, if the traversal symbol indicates an angle (angle in S114), the vertex is encoded using three angles (S116).
  • the traversal symbol indicates a parallelogram (parallelogram in S114)
  • the vertex is encoded using a parallelogram (S117).
  • the traversal symbol indicates a polynomial (polynomial in S114)
  • the vertex is encoded using the polynomial (S118).
  • the optimal processing mode may be determined by the minimum cost among the multiple costs calculated based on the encoding rate and distortion for the multiple processing modes. Coding rate and distortion costs may be calculated based on the number of bits signaled and the error between the actual and predicted values. Also, based solely on the error between the actual value and the predicted value, the optimal processing mode corresponding to the smallest error may be selected.
  • a traversal symbol indicating any of skip, end, and execution may be encoded.
  • a processing mode indicating either coordinates, angle, parallelogram, or polygon may be encoded separately from the traversal symbol. Note that if the processing mode is determined in advance, the processing mode does not need to be encoded or the processing does not need to be switched.
  • the first triangle has a first apex A, a second apex B, and a third apex C
  • the second triangle has a second apex B.
  • the corresponding triangle has a second apex B, a third apex C and a fifth apex E.
  • FIG. 31 is a conceptual diagram showing examples of three angles according to this aspect.
  • the two interior angles of the corresponding triangles are used as the second angle ⁇ and the third angle ⁇ .
  • two interior angles of corresponding triangles touch a common side.
  • the second angle ⁇ and the third angle ⁇ are calculated by the following equation (1).
  • FIG. 32 is a conceptual diagram showing another example of three angles according to this aspect.
  • the angle that touches the line AE is used as the second angle ⁇
  • one interior angle of the corresponding triangle that touches the common side BC is used as the third angle ⁇ .
  • the second angle ⁇ and the third angle ⁇ are calculated by the following equation (2).
  • FIG. 33 is a conceptual diagram showing still another example of three angles according to this aspect.
  • one interior angle of the corresponding triangle is used as the second angle ⁇
  • the angle between AB and AQ is used as the third angle ⁇ .
  • Point Q is the orthogonal projection of the fifth vertex E onto the common edge BC.
  • the second angle ⁇ and the third angle ⁇ are calculated by the following equation (3).
  • the dihedral angle between the planes of the first triangle and the second triangle is used as the first angle ⁇ .
  • the first angle ⁇ may be calculated using those surface normals.
  • FIG. 34 is a syntax diagram showing an example of the syntax structure of vertex information for a plurality of vertices according to this aspect.
  • a traversal symbol is encoded for each other vertex. Then, it is controlled whether or not to encode vertices according to the traversal symbol.
  • FIG. 35 is a syntax diagram showing an example of the syntax structure of vertex information for one vertex according to this aspect.
  • prediction_mode is equal to only_angles_mode, the derived angles for the first, second, and third angles are directly encoded.
  • FIG. 36 is a syntax diagram showing another example of the syntax structure of vertex information for one vertex according to this aspect.
  • is_init_angles other methods are used to derive or predict the three angles without directly encoding them.
  • An example of how to derive or predict the three angles is to use a historical list of previously encoded angles. Specifically, if the three angles to be encoded are not the first three angles, the three angles are derived or predicted using a history list of previously encoded angles.
  • FIG. 37 is a table diagram showing an example of a table used for angle prediction.
  • previously encoded angles and indices are stored in a table in association with each other.
  • the angles stored in the table may be calculated by averaging three previously encoded angles.
  • the angles stored in the table may be controlled according to first in, first out, frequency of occurrence of values, or a combination of both.
  • the index may be a fixed value.
  • one of the multiple angles in the table is used as the predicted angle.
  • the index value corresponding to the predicted angle and the delta which is the difference between the actual angle and the predicted angle, are written to the bitstream.
  • FIG. 38 is a table diagram showing another example of a table used for angle prediction.
  • the previously encoded first angle, second angle, and third angle are stored in the table in association with the index.
  • the angles stored in the table may be controlled by first in, first out, frequency of occurrence of values, or a combination of both.
  • the index may be a fixed value.
  • one combination of the plurality of combinations in the table is used as a combination of predicted angles.
  • the index value corresponding to the combination of predicted angles and the delta which is the difference between the actual angle and the predicted angle, are written to the bitstream.
  • FIG. 39 is a conceptual diagram showing an example of prediction using angles with respect to the vertices of the previous triangle.
  • the three angles for the previous triangle's vertices are used to derive or predict the three angles for the current triangle's vertices.
  • the previous triangle is the triangle that is traversed before the current triangle.
  • the previous triangle may be the triangle immediately before the current triangle, or may be the triangle adjacent to the current triangle.
  • three angles (10°, 30°, 60°) to the vertices of the previous triangle are used to determine the syntax values corresponding to the three angles to the vertices of the current triangle.
  • FIG. 40 is a conceptual diagram showing an example of prediction using angles with respect to the vertices of the previous three-dimensional mesh.
  • three angles are derived or predicted using angles for vertices of a previous three-dimensional mesh.
  • the previous three-dimensional mesh is a three-dimensional mesh that is spatially different from the current three-dimensional mesh, and is a three-dimensional mesh that was processed before the current three-dimensional mesh.
  • the previous three-dimensional mesh is a three-dimensional mesh that is temporally different from the current three-dimensional mesh, and is a three-dimensional mesh that was processed before the current three-dimensional mesh.
  • the first angles, the second angles, and the third angles for the vertices of the previous three-dimensional mesh may be averaged and signaled in the current three-dimensional mesh.
  • the syntax value 1st delta 1st angle x - average 1st angle is determined and written.
  • this method may be utilized in dynamic meshes where adjacent frames have spatial redundancy.
  • FIG. 41 is a block diagram showing a configuration example of the encoding device 100 according to this aspect.
  • the encoding device 100 encodes a three-dimensional mesh into a bitstream.
  • the encoding device 100 may include a traverser 161, a switch 162, an angle derivator 163, an angle encoder 164, a coordinate encoder 165, and an entropy encoder 166.
  • the traverser 161 acquires a three-dimensional mesh. Then, the traverser 161 determines a vertex to be encoded in the three-dimensional mesh, and outputs the vertex to either the angle derivator 163 or the coordinate encoder 165 via the switch 162. In one example, traverser 161 selects coordinate encoder 165 for the first three vertices of the three-dimensional mesh and selects angle derivator 163 for other vertices.
  • the coordinate encoder 165 takes in three vertices forming a first triangle on the first plane and encodes the three vertices.
  • Delta encoding may be used in encoding the three vertices. Specifically, the coordinate value of the first vertex A is directly encoded, and the difference between the coordinate value of the first vertex A and the coordinate value of the second vertex B, and the coordinate value of the first vertex A and the third vertex B are encoded. The difference from the coordinate value of C may be encoded.
  • the angle deriver 163 takes in the fourth vertex D that forms a second triangle on the second plane using the second vertex B and the third vertex C, and derives three angles for specifying the fourth vertex D. and outputs the three derived angles to the angle encoder 164.
  • the second plane may be different from the first plane or may be the same as the first plane. As mentioned above, Figures 31, 32 and 33 show different examples for three angles.
  • the angle encoder 164 encodes three angles. Additional residuals may be calculated and signaled separately after the angle signaling. In the example of fully representing the angular floating point value for lossless encoding, the angular encoder 164 encodes the integer portion of the angular floating point value and then encodes the fractional portion of the angular floating point value as a residual. It may also be encoded as a value.
  • the encoded information obtained by the coordinate encoder 165 and the encoded information obtained by the angle encoder 164 are sent to the entropy encoder 166 and also fed back to the traverser 161.
  • Entropy encoder 166 compresses the encoded information and outputs a bitstream.
  • Traverser 161 uses the encoded information to determine the next traversal.
  • Examples of encoding methods to which entropy encoding applies are Huffman encoding, arithmetic encoding, range encoding, ANS (Asymmetric Numeral Systems), or CABAC (Context Adaptive Binary Arithmetic Coding). - Adaptive Binary Arithmetic Coding).
  • the encoding device 100 encodes a three-dimensional mesh according to the above configuration and processing. As a result, the amount of codes of the three-dimensional mesh may be suppressed.
  • the corresponding triangle is a triangle that is congruent to the second triangle, but may be a triangle that is similar to the second triangle. Even if the second triangle and the corresponding triangle are similar, if the correspondence between the second triangle and the corresponding triangle is known, the fifth apex E to the fourth apex will be moved according to the correspondence between the second triangle and the corresponding triangle. D can be derived. Further, the corresponding triangle may be an orthogonal projection of the second triangle onto the first plane. In this case as well, the fourth vertex D can be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle.
  • FIG. 42 is a flowchart illustrating an example of decoding processing according to this aspect.
  • a first vertex, a second vertex, and a third vertex are derived from the three-dimensional mesh (S201).
  • the first vertex, the second vertex, and the third vertex form a first triangle on the first plane of the three-dimensional mesh.
  • the first vertex, the second vertex, and the third vertex are three decoded vertices.
  • the first angle, the second angle, and the third angle are bits for deriving the fourth vertex of the second triangle formed on the second plane of the three-dimensional mesh using the common side with the first triangle.
  • the stream is decoded (S202).
  • the fourth vertex is derived using the first angle, the second angle, and the third angle (S203).
  • FIG. 43 is a flowchart showing a specific example of the decoding process according to this aspect.
  • the decoding process begins by decoding each vertex coordinate value of the initially encoded initial surface (S211). Subsequently, the traversal symbols of the current plane are decoded (S212). Then, it is determined whether the traversal symbol indicates coordinates, angle, skip, or end (S213).
  • the traversal symbol indicates coordinates (coordinates in S213)
  • the vertex coordinate values (x, y, z) are directly decoded (S214).
  • the traversal symbol indicates an angle (angle in S213)
  • the vertex is decoded using three angles (S215).
  • the traversal symbol indicates skip (skip in S213)
  • the process is repeated for the next side.
  • the traversal symbol indicates the end (end in S213), the process ends.
  • a traversal symbol indicating any of skip, end, and execution may be decoded.
  • Skip corresponds to the vertex being already processed.
  • Termination corresponds to all vertices being processed.
  • the execution corresponds to the vertex to be processed.
  • a traversal symbol indicating execution is decoded, a processing mode indicating either coordinates or angle may be decoded separately from the traversal symbol. Processing mode may also be expressed as mode, operation mode, prediction mode, encoding mode, or decoding mode.
  • FIG. 44 is a flowchart showing another specific example of the decoding process according to this aspect.
  • the decoding process of the reference example is integrated with the decoding process of FIG.
  • the decoding process begins by decoding each vertex coordinate value of the initially encoded initial surface (S211). Subsequently, the traversal symbols of the current plane are decoded (S212). Then, it is determined whether the traversal symbol indicates coordinates, angle, skip, or end (S213).
  • the traversal symbol indicates coordinates (coordinates in S213)
  • the vertex coordinate values (x, y, z) are directly decoded (S214).
  • the traversal symbol indicates an angle (angle in S213)
  • the vertex is decoded using three angles (S215).
  • the traversal symbol indicates a parallelogram (parallelogram in S213)
  • the vertex is decoded using the parallelogram (S216).
  • the traversal symbol indicates a polynomial (polynomial in S213)
  • the next vertex is decoded using the polynomial.
  • the process is repeated for the next side. Further, if the traversal symbol indicates the end (end in S213), the process ends.
  • a traversal symbol indicating any of skip, end, and execution may be decoded.
  • a processing mode indicating either coordinates, angle, parallelogram, or polygon may be decoded separately from the traversal symbol. Note that if the processing mode is determined in advance, the processing mode may not be decoded or the processing may not be switched.
  • FIG. 34 is a syntax diagram showing an example of the syntax structure of vertex information for a plurality of vertices according to this aspect.
  • traversal symbols are decoded for each other vertex. Then, it is controlled whether or not to decode the vertex according to the traversal symbol.
  • FIG. 35 is a syntax diagram showing an example of the syntax structure of vertex information for one vertex according to this aspect.
  • prediction_mode is equal to only_angles_mode, the derived angles for the first, second, and third angles are directly decoded.
  • FIG. 36 is a syntax diagram showing another example of the syntax structure of vertex information for one vertex according to this aspect.
  • is_init_angles other methods are used to derive or predict the three angles without directly decoding them.
  • An example of how to derive or predict the three angles is to use a historical list of previously decoded angles. Specifically, if the three angles to be decoded are not the first three angles, a history list of previously decoded angles is used to derive or predict the three angles.
  • FIG. 37 is a table diagram showing an example of a table used for angle prediction.
  • previously decoded angles and indexes are stored in a table in association with each other.
  • the angles stored in the table may be calculated by averaging three previously decoded angles.
  • the angles stored in the table may be controlled according to first in, first out, frequency of occurrence of values, or a combination of both.
  • the index may be a fixed value.
  • one of the multiple angles in the table is used as the predicted angle.
  • the syntax elements only the index value corresponding to the predicted angle and the delta, which is the difference between the actual angle and the predicted angle, are decoded from the bitstream.
  • FIG. 38 is a table diagram showing another example of a table used for angle prediction.
  • the previously decoded first angle, second angle, and third angle are stored in the table in association with the index.
  • the angles stored in the table may be controlled according to first in, first out, frequency of occurrence of values, or a combination of both.
  • the index may be a fixed value.
  • one combination of the plurality of combinations in the table is used as a combination of predicted angles.
  • the syntax elements only the index values corresponding to the combination of predicted angles and the delta, which is the difference between the actual angle and the predicted angle, are decoded from the bitstream.
  • FIG. 39 is a conceptual diagram showing an example of prediction using angles with respect to the vertices of the previous triangle.
  • the three angles for the previous triangle's vertices are used to derive or predict the three angles for the current triangle's vertices.
  • the previous triangle is the triangle that is traversed before the current triangle.
  • the previous triangle may be the triangle immediately before the current triangle, or may be the triangle adjacent to the current triangle.
  • three angles (10°, 30°, 60°) to the vertices of the previous triangle are used to determine the syntax values corresponding to the three angles to the vertices of the current triangle.
  • FIG. 40 is a conceptual diagram showing an example of prediction using angles with respect to the vertices of the previous three-dimensional mesh.
  • three angles are derived or predicted using angles for vertices of a previous three-dimensional mesh.
  • the previous three-dimensional mesh is a three-dimensional mesh that is spatially different from the current three-dimensional mesh, and is a three-dimensional mesh that was processed before the current three-dimensional mesh.
  • the previous three-dimensional mesh is a three-dimensional mesh that is temporally different from the current three-dimensional mesh, and is a three-dimensional mesh that was processed before the current three-dimensional mesh.
  • Additional residuals may be decoded after decoding the three angles.
  • the value decoded as an angle may represent the integer portion of the floating point value of the angle
  • the value decoded as the residual may represent the fractional portion of the floating point value of the angle. By adding the decimal part to the integer part, the actual angle is obtained.
  • the first triangle has a first apex A, a second apex B, and a third apex C
  • the second triangle has a second apex B.
  • the corresponding triangle has a second apex B, a third apex C and a fifth apex E.
  • the fifth vertex E which is the projection of the fourth vertex D onto the first plane containing the first triangle, is calculated using the second and third angles. . Then, the fifth vertex E to the fourth vertex D are calculated using the first angle that is the dihedral angle between the first triangle and the second triangle.
  • FIG. 31 is a conceptual diagram showing examples of three angles according to this aspect.
  • the two interior angles of the corresponding triangles are used as the second angle ⁇ and the third angle ⁇ . That is, in this example, the two interior angles of the corresponding triangles are decoded from the bitstream.
  • the following equation (4) is obtained.
  • FIG. 32 is a conceptual diagram showing another example of three angles according to this aspect.
  • the angle that touches the line AE is used as the second angle ⁇
  • one interior angle of the corresponding triangle that touches the common side BC is used as the third angle ⁇ .
  • the fourth angle ⁇ between BA and BC is derived by the following equation (5).
  • the magnitude of BE is derived from the above equation (6). Therefore, the direction and size of BE and the plane on which the fifth vertex E is located are known, and the coordinate values of the fifth vertex E are derived.
  • FIG. 33 is a conceptual diagram showing still another example of three angles according to this aspect.
  • one interior angle of the corresponding triangle is used as the second angle ⁇
  • the angle between AB and AQ is used as the third angle ⁇ .
  • Point Q is the orthogonal projection of the fifth vertex E onto the common edge BC.
  • the magnitude of CE is derived from equation (8) above. Therefore, the direction and size of CE and the plane on which the fifth vertex E is located are known, and the coordinate values of the fifth vertex E are derived.
  • the fifth vertex E is derived in each of the plurality of derivation methods explained using FIGS. 31, 32, and 33.
  • the fourth vertex D can be derived from the fifth vertex E.
  • the second triangle CBD and the corresponding triangle CBE are the same on a one-to-one basis.
  • the fourth vertex D and the fifth vertex E are on different planes that share a common side CB.
  • the fourth vertex D is derived by rotating the fifth vertex E around the side BC by an amount of ⁇ - ⁇ (or ⁇ ).
  • ⁇ is the first angle.
  • FIG. 45 is a table diagram showing an example of a lookup table for a sine function and a cosine function. In calculating the above sine function and cosine function, the lookup table shown in FIG. 45 may be used.
  • FIG. 46 is a block diagram showing a configuration example of a decoding device 200 according to this aspect.
  • Decoding device 200 decodes a three-dimensional mesh from a bitstream.
  • the decoding device 200 includes a traverser 261, a switch 262, an angle decoder 263, a coordinate deriver 264, a coordinate decoder 265, and an entropy decoder 266.
  • the entropy decoder 266 obtains the bitstream. Entropy decoder 266 obtains decompression information by decompressing the bitstream and outputs the decompression information. Examples of encoding methods to which entropy decoding applies are Huffman encoding, arithmetic encoding, range encoding, ANS (Asymmetric Numeral Systems), or CABAC (Context Adaptive Binary Arithmetic Coding). Adaptive Binary Arithmetic Coding).
  • the decompression information is input to the traverser 261.
  • Traverser 261 interprets the decompression information and outputs the decompression information to either angle decoder 263 or coordinate decoder 265 via switch 262 .
  • the coordinate decoder 265 decodes the three vertices forming the first triangle on the first plane from the decompression information. As an example of decoding three vertices, the coordinate value of the first vertex A and two difference values are decoded from the decompression information. Next, by adding two difference values to the coordinate value of the first vertex A, the coordinate value of the second vertex B and the coordinate value of the third vertex C are derived.
  • the angle decoder 263 decodes the three angles from the decompression information and outputs the three angles to the coordinate derivator 264.
  • the coordinate deriver 264 uses the three angles to derive the vertex of the second triangle that has a common side with the first triangle.
  • the second triangle may be on a plane different from the first plane, or may be on the same plane as the first plane. Specifically, the coordinate deriver 264 derives the vertices of the second triangle from the three angles using the derivation method described above with reference to FIGS. 31, 32, and 33.
  • the vertex information indicating the vertices obtained by the coordinate deriver 264 or the coordinate decoder 265 is output as vertex information of the three-dimensional mesh. Further, this vertex information is fed back to the traverser 261 and used to determine the next traversal.
  • the decoding device 200 decodes the three-dimensional mesh according to the above configuration and processing. As a result, the amount of codes of the three-dimensional mesh may be suppressed.
  • the corresponding triangle is a triangle that is congruent to the second triangle, but may be a triangle that is similar to the second triangle. Even if the second triangle and the corresponding triangle are similar, if the correspondence between the second triangle and the corresponding triangle is known, the fifth apex E to the fourth apex will be moved according to the correspondence between the second triangle and the corresponding triangle. D can be derived. Further, the corresponding triangle may be an orthogonal projection of the second triangle onto the first plane. In this case as well, the fourth vertex D can be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle.
  • the vertex information encoding process in the present disclosure is applicable to encoding point position information in point cloud compression methods such as V-PCC (Point Cloud Compression) and G-PCC (Geometry-based Point Cloud Compression), for example. It is.
  • the encoding device 100 encodes information for specifying the fourth vertex D of the second triangle from the reference first triangle or its three vertices, which are the first vertex A, the second vertex B, and the third vertex C. become
  • the decoding device 200 decodes information for specifying the fourth vertex D of the second triangle from the reference first triangle or its three vertices, which are the first vertex A, the second vertex B, and the third vertex C. .
  • the information for specifying the fourth vertex D includes dihedral angle information indicating the dihedral angle and specific information for specifying the fifth vertex E.
  • the dihedral angle indicated by the dihedral angle information is a dihedral angle formed by a first plane containing the first triangle and a second plane containing the second triangle.
  • the fifth vertex E specified by the specific information is a vertex on the first plane, and is a vertex of a corresponding triangle corresponding to the second triangle.
  • the fifth vertex E is a vertex on the first plane, it can be relatively easily identified from the first triangle on the first plane. Further, the fourth vertex D can be derived from the fifth vertex E according to the correspondence between the second triangle and the corresponding triangle and the dihedral angle formed between the first plane and the second plane. Since the dihedral angle ranges from -180 degrees to 180 degrees, for example, an increase in the amount of code is suppressed. Therefore, with the above configuration and processing, the fourth vertex D can be efficiently encoded and decoded.
  • the information for specifying the fourth vertex D may be used as the above-mentioned specific information or in place of the above-mentioned specific information, such as "(1) of the vertex (fifth vertex E) of the polygon (corresponding triangle BEC)
  • the information may include "position information” or "(2) information regarding the position and shape of the polygon (corresponding triangle BEC)".
  • the information for specifying the fourth vertex D may be provided as the dihedral angle information described above or in place of the dihedral angle information described above, such as "(3) Vertex (fifth "Information indicating the distance and direction between the position of the vertex E) and the position of the processing target vertex (fourth vertex D)" may be included. Further, the information for specifying the fourth vertex D may be used as the above-mentioned dihedral angle information or in place of the above-mentioned dihedral angle information as "(4) a plane containing a polygon (corresponding triangle BEC),” "information indicating a difference from a plane including a polygon (second triangle BDC) having a vertex to be processed (fourth vertex D)".
  • any of the above information may be indicated using angles. This may reduce the amount of code and improve the flexibility and accuracy of prediction. Further, information indicating which of the above information is used may be encoded.
  • the following example may be added to the above configuration and processing, or at least a part of the above configuration and processing may be replaced with the following example.
  • Position information of vertices of a polygon may indicate the coordinate values of the second vertex B, the third vertex C, and the fifth vertex E of the corresponding triangle BEC. If the information regarding the position and shape of the first reference triangle or the position information of the first vertex A, the second vertex B, and the third vertex C are specified, the position information of the vertices of the polygon is the fifth vertex. Only the coordinate value of E may be shown.
  • the coordinate value of the fifth vertex E may be expressed by two-dimensional coordinate values.
  • the coordinate value may be expressed using a difference with other coordinate values, and the coordinate value of the fifth vertex E indicates the difference with respect to the coordinate value of the first vertex A, the second vertex B, or the third vertex C. It may also be expressed using a vector.
  • the polygon may be other than a triangle and may be a quadrilateral.
  • Information regarding the position and shape of the polygon may be information that specifies the position and shape of the corresponding triangle BEC.
  • Information regarding the position and shape of the first reference triangle ABC, or if the position information of the first vertex A, second vertex B, and third vertex C is specified, information specifying the position and shape of the corresponding triangle BEC. may indicate information indicating the side BC, the angle between the side BC and the side CE, and the angle between the side BC and the side BE.
  • the information indicating the side BC corresponds to the information indicating that the corresponding triangle BEC is used.
  • information indicating the first vertex A may be encoded.
  • the information indicating the first vertex A corresponds to the information indicating that the fifth vertex E is on the opposite side of the first vertex A across the side BC.
  • a corresponding triangle BEC on the same plane as the first triangle ABC is to be used for deriving the positional information of the fourth vertex D to be processed, only the above two angles with respect to the fifth vertex E are used. It may also be encoded. This makes it possible to derive the position and shape of the corresponding triangle BEC, that is, the position information of the fifth vertex E, from only two angles, compared to the case where the coordinate values of the fifth vertex E are encoded. It becomes possible to reduce the amount of code.
  • the angle itself may be encoded, or an index specifying one angle among a plurality of predefined angles may be encoded. Multiple combinations of two angles may be predefined. Then, an index identifying one combination among the plurality of combinations may be encoded.
  • information indicating one angle and the length of one side may be encoded.
  • the angle formed by side BC and side BE and the length of side BE may be encoded.
  • information indicating one angle and the length of one perpendicular line may be encoded.
  • the angle between sides BC and BE and the length of a perpendicular drawn from the fifth vertex E to side BC may be encoded.
  • information indicating the lengths of the two sides may be encoded.
  • the lengths of sides BE and CE may be encoded.
  • “(3) Information indicating the distance and direction between the position of the polygon's vertex and the position of the processing target vertex” is the distance between the fifth vertex E of the corresponding triangle BEC and the fourth vertex D to be processed. and a difference vector indicating the direction. If the position of the fourth vertex D matches the position of the fifth vertex E, or if the fourth vertex D is located on the same plane as the fifth vertex E, this information may be omitted.
  • "(4) Information indicating the difference between the plane containing the polygon and the plane containing the polygon having the processing target apex" is the difference between the plane containing the corresponding triangle BEC and the plane containing the second triangle BDC.
  • An angle may also be indicated.
  • the angle for example, the angle between the side BE and the side BD, or the angle between the side CE and the side CD may be encoded.
  • the angle itself may be encoded as the angle between the side BE and the side BD or the angle between the side CE and the side CD, or an index that specifies one angle among a plurality of predefined angles. may be encoded.
  • the plurality of predefined angles may also be used in common when deriving the angle between side BC and side CE and the angle between side BC and side BE.
  • an index indicating a plurality of combinations of angles formed by sides BE and BD, angles formed by sides CE and sides CD, angles formed by sides BC and sides CE, and angles formed by sides BC and sides BE. may be defined in advance. This makes it possible to further reduce the amount of code compared to the case where the angle itself is encoded.
  • the angle may be omitted. Furthermore, in cases where the angle is expressed by an index, the angle is expressed with integer precision, or the coordinate value is expressed with integer precision, a difference vector may be encoded in addition to the angle. Thereby, it becomes possible to modify the derived coordinate value of the fourth vertex D, and it becomes possible to improve the accuracy of encoding.
  • the fourth vertex D may be used as the first vertex A, the second vertex B, or the third vertex C with respect to other vertices.
  • a plurality of processing methods may be switched by selecting a processing method such as an encoding method or a decoding method for each processing target vertex.
  • information indicating the processing method used to encode the processing target vertex may be encoded.
  • a plurality of processing methods may be switched by selecting a processing method for each three-dimensional mesh, or information indicating the processing method may be encoded for each three-dimensional mesh.
  • the difference described in the present disclosure may be expressed as a deviation, a residual, or a delta.
  • the apex described in this disclosure may mean the position of the apex.
  • the corresponding triangles in the above description may be expressed as predicted triangles, or may be expressed as virtual corresponding triangles.
  • the fifth vertex E of the corresponding triangle may be expressed as a corresponding vertex, a corresponding point, a predicted vertex, or a predicted point.
  • the second aspect of encoding and decoding vertex information is an aspect that can be combined with the first aspect of encoding and decoding vertex information.
  • 3D graphical data captured with various real-world hardware tends to have irregularities such as degenerate surfaces, duplicate vertices, and unconnected parts.
  • a degenerate surface occurs when at least two of the vertices that make up the surface have the same vertex index, so the surface area becomes zero and the surface normal becomes indeterminate.
  • Duplicate vertices have exactly the same coordinates, but are expressed with two or more different indexes. Therefore, the same point is represented by different indexes on different surfaces. Therefore, in traversing the topology of the mesh, these vertices are considered non-identical, leading to the failure of encoding techniques that rely on using previously encoded vertices. Additionally, the polygon mesh may have unconnected or incomplete parts, making it difficult to traverse the entire model.
  • virtual reference triangles are generated that allow the implementation of angle-based 3D graphics encoding techniques.
  • the virtual reference triangle may be constructed using a combination of previously encoded vertices and dummy vertices selected according to predetermined conditions. Exception processing for the 3D model may be instructed from the encoding device 100 to the decoding device 200 using a mode setting parameter.
  • FIG. 47 is a flowchart illustrating an example of encoding processing according to this aspect.
  • connection information of the first vertex, the second vertex, and the third vertex is derived from the three-dimensional mesh (S121).
  • the first vertex, the second vertex, and the third vertex form a first triangle.
  • connection information satisfies the conditions (S122).
  • the following two types of conditions are possible.
  • the first condition is that the first vertex, the second vertex, and the third vertex are connected to each other.
  • the second condition is that the first vertex, the second vertex, and the third vertex have already been encoded. Note that not only connection information but also vertex information may be derived and used for these determinations.
  • FIG. 48 is a conceptual diagram showing an example of a mesh according to this aspect.
  • A1, B1, C1, D1, E1 and F1 are vertices in the geometry map.
  • FIG. 49 is a conceptual diagram showing an example of a fragmented mesh in which some of the three vertices forming a triangle are not connected.
  • A2, B2, C2, D2, E2 and F2 are vertices in the connectivity map.
  • solid lines indicate connections and dotted lines indicate non-connections. Note that portions without lines are also non-connected.
  • vertex D1 of triangle A1B1C1 is not suitable for use since there is no connectivity between A2 and C2 and between A2 and B2.
  • FIG. 50 is a conceptual diagram showing an example of a mesh with overlapping vertices, and shows an example in which the vertices of the reference triangle are not encoded before the encoding target vertices are encoded.
  • Vertex C and vertex F are two different points in the mesh and exist on different faces in the mesh. Since apex C and apex F have the same coordinates, they are displayed as overlapping points.
  • vertexs A, B, and C are encoded in traversal order.
  • Vertex D is then encoded in an angular manner using reference triangle ABC.
  • vertex E has not yet been encoded.
  • virtual vertices and virtual triangles are sometimes used in a 3D mesh to efficiently represent a 3D image, and even in such cases, the vertices may not be encoded or the connectivity between the vertices may be poor. There is a possibility that there is no such thing.
  • the first vertex, the second vertex, and the third vertex are derived as three vertices (S123).
  • the second vertex, the third vertex, and the fourth vertex form a second triangle on the three-dimensional mesh, and the fourth vertex is on the opposite side of the first vertex.
  • FIG. 31 shows an example of the positions of three vertices, where B and C are the second and third vertices, A is the first vertex, and D is the fourth vertex.
  • a reference triangle formed by ABC may be used to encode vertex D.
  • the fifth vertex is different from the first vertex, the second vertex, and the third vertex. Note that the fifth vertex in this aspect is a vertex used in place of any one of the first vertex, the second vertex, and the third vertex, and has a different meaning from the fifth vertex in the first aspect.
  • FIG. 51 is a conceptual diagram showing an example of exception handling using a previously encoded vertex set. This example is an example where vertex B does not satisfy the condition (does not exist or has not been encoded yet).
  • the BCD reference triangle cannot be used to encode vertex E.
  • Vertex B is replaced by the previously encoded vertex A (the fifth vertex), forming a new virtual reference triangle ACD that can be used to encode vertex E.
  • the fifth vertex may be created as a dummy vertex with predefined coordinates.
  • a virtual reference triangle may be formed including dummy vertices.
  • the encoding device 100 may calculate dummy vertices.
  • dummy vertices are predefined at fixed coordinates (1, 0, 0), (0, 1, 0), and (0, 0, 1), and these dummy vertices are the first vertex, the second vertex, and the second vertex. It may also play the role of three vertices.
  • the coordinates of the dummy vertices may be selected in a model-specific manner.
  • encoding device 100 may calculate the dimensions of the object's 3D bounding box and use those values to create dummy vertices.
  • the created dummy vertices may be signaled in the bitstream.
  • the same number of dummy vertices or vertices encoded in the past as the number of vertices that do not satisfy the condition may be used.
  • a plurality of angles for specifying the fourth vertex are derived from the triangle formed by the derived three vertices (S125).
  • the fourth vertex forms a second triangle using a common side with the triangle formed based on the derived three vertices.
  • the number of angles derived depends on the prediction method used to encode the fourth vertex. 31, 32, and 33 show different examples of deriving at least a plurality of angles using the fourth vertex and the three derived vertices A, B, and C.
  • the multiple angles are encoded into a bitstream (S126).
  • An example of encoding multiple angles is using entropy encoding.
  • Examples of coding schemes to which entropy coding applies may be Huffman coding, arithmetic coding, range coding, asymmetric number system (ANS), or context adaptive binary arithmetic coding (CABAC).
  • FIG. 52 is a flowchart showing a specific example of encoding processing according to this aspect.
  • a process (S131) for encoding a vertex using an exception is added.
  • This process (S131) corresponds to the process (S124, S125, S126) shown in FIG. 47.
  • the process of encoding vertices using exceptions is one of the encoding modes used for compressing mesh data.
  • FIG. 53 is a block diagram showing a configuration example of the encoding device 100 according to this aspect.
  • the encoding device 100 encodes a three-dimensional mesh into a bitstream.
  • the encoding device 100 may include a traverser 161, a determiner 167, an exception handler 168, an angle derivator 163, and an entropy encoder 166.
  • the traverser 161 acquires a three-dimensional mesh. Further, the traverser 161 outputs connection information of the first vertex, the second vertex, and the third vertex derived from the three-dimensional mesh to the determiner 167. The traverser 161 also determines the fourth vertex to be encoded in the three-dimensional mesh.
  • the first vertex, the second vertex, and the third vertex form a first triangle, and have a common side with the second triangle including the fourth vertex.
  • the determiner 167 checks whether the connection information satisfies the conditions. If it is determined that the connection information satisfies the condition, the angle deriving unit 163 performs processing using the first vertex, the second vertex, and the third vertex.
  • the exception handler 168 derives three vertices including at least the fifth vertex.
  • the angle derivator 163 performs processing using three vertices including at least the fifth vertex.
  • the angle derivator 163 derives multiple angles using the three vertices and the fourth vertex.
  • the vertices represent geometric coordinates
  • three angles are derived.
  • the number of angles derived depends on the prediction method used to encode the fourth vertex.
  • the entropy encoder 166 encodes the multiple angles derived by the angle derivator 163 into a bitstream.
  • FIG. 54 is a flowchart illustrating an example of decoding processing according to this aspect.
  • connection information of the first vertex, the second vertex, and the third vertex is derived from the bitstream (S221).
  • the first vertex, the second vertex, and the third vertex form a first triangle.
  • connection information satisfies the conditions (S222).
  • the following two types of conditions are possible.
  • the first condition is that the first vertex, the second vertex, and the third vertex are connected to each other.
  • the second condition is that the first vertex, the second vertex, and the third vertex have already been decoded. Note that not only connection information but also vertex information may be derived and used for these determinations.
  • FIG. 48 is a conceptual diagram showing an example of a mesh according to this aspect.
  • A1, B1, C1, D1, E1 and F1 are vertices in the geometry map.
  • FIG. 49 is a conceptual diagram showing an example of a fragmented mesh in which some of the three vertices forming a triangle are not connected.
  • A2, B2, C2, D2, E2 and F2 are vertices in the connectivity map.
  • solid lines indicate connections and dotted lines indicate non-connections. Note that portions without lines are also non-connected.
  • vertex A1 of triangle A1B1C1 is not suitable for use since there is no connectivity between A2 and C2 and between A2 and B2.
  • FIG. 50 is a conceptual diagram showing an example of a mesh with overlapping vertices, and shows an example in which the vertices of the reference triangle are not decoded before the decoding target vertex is decoded.
  • Vertex C and vertex F are two different points in the mesh and exist on different faces in the mesh. Since apex C and apex F have the same coordinates, they are displayed as overlapping points.
  • vertexs A, B, and C are decoded in traversal order.
  • Vertex D is then decoded in an angular manner using reference triangle ABC.
  • vertex E has not yet been decoded.
  • virtual vertices and virtual triangles are sometimes used in a 3D mesh to efficiently represent a 3D image, and even in such cases, vertices may not be decoded or there may be connectivity between vertices. There is a possibility that there is no such thing.
  • the first vertex, the second vertex, and the third vertex are derived as three vertices (S223).
  • the second vertex, the third vertex, and the fourth vertex form a second triangle on the three-dimensional mesh, and the fourth vertex is on the opposite side of the first vertex.
  • FIG. 31 shows an example of the positions of three vertices, where B and C are the second and third vertices, A is the first vertex, and D is the fourth vertex.
  • a reference triangle formed by ABC can be used to decode vertex D.
  • the fifth vertex is different from the first vertex, the second vertex, and the third vertex. Note that the fifth vertex in this aspect is a vertex used in place of any one of the first vertex, the second vertex, and the third vertex, and has a different meaning from the fifth vertex in the first aspect.
  • FIG. 51 is a conceptual diagram showing an example of exception handling using a previously decoded vertex set. This example is an example where vertex B does not satisfy the condition (does not exist or has not been decoded yet).
  • the BCD reference triangle cannot be used to decode vertex E.
  • Vertex B is replaced by the previously decoded vertex A (the fifth vertex), forming a new virtual reference triangle ACD that can be used to decode vertex E.
  • the fifth vertex may be created as a dummy vertex with predefined coordinates.
  • a virtual reference triangle may be formed including dummy vertices.
  • the decoding device 200 may calculate dummy vertices similarly to the encoding device 100.
  • dummy vertices are predefined at fixed coordinates (1, 0, 0), (0, 1, 0), and (0, 0, 1), and these dummy vertices are the first vertex, the second vertex, and It may also play the role of a third vertex.
  • the coordinates of the dummy vertices may be selected in a model-specific manner.
  • decoding device 200 may calculate the dimensions of the object's 3D bounding box and use those values to create dummy vertices.
  • the dummy vertices created by the encoding device 100 may be signaled in a bitstream and decoded by the decoding device 200.
  • the same number of dummy vertices or vertices that have been decoded in the past as the number of vertices that do not satisfy the condition may be used.
  • multiple angles are decoded from the bitstream (S225).
  • An example of decoding multiple angles is to use entropy decoding.
  • Examples of coding schemes that entropy decoding follows may be Huffman coding, arithmetic coding, range coding, asymmetric number system (ANS), or context adaptive binary arithmetic coding (CABAC). The number of angles that are decoded depends on the prediction method used to decode the fourth vertex.
  • a fourth vertex is derived using multiple angles (S226).
  • the fourth vertex forms a second triangle using a common side with the triangle formed based on the derived three vertices.
  • 31, 32, and 33 show different examples of deriving the fourth vertex using the three derived vertices A, B, and C and a plurality of angles.
  • FIG. 55 is a flowchart showing a specific example of decoding processing according to this aspect.
  • a process (S231) for decoding a vertex using an exception is added.
  • This process (S231) corresponds to the process (S224, S225, S226) shown in FIG.
  • the process of decoding vertices using exceptions is one of the decoding modes used to decompress a bitstream.
  • FIG. 56 is a block diagram showing a configuration example of a decoding device 200 according to this aspect.
  • Decoding device 200 decodes a three-dimensional mesh from a bitstream.
  • the decoding device 200 includes an entropy decoder 266, a traverser 261, a determiner 267, an exception handler 268, and a coordinate deriver 264.
  • the entropy decoder 266 obtains the bitstream. Furthermore, the entropy decoder 266 decodes connection information of the first vertex, the second vertex, and the third vertex, and the plurality of angles from the bitstream. Here, the first vertex, the second vertex, and the third vertex form a first triangle.
  • the traverser 261 outputs the connection information decoded by the entropy decoder 266 to the determiner 267.
  • the determiner 267 checks whether the connection information satisfies the conditions. If it is determined that the connection information satisfies the conditions, the coordinate derivation unit 264 performs processing using the first vertex, the second vertex, and the third vertex.
  • the exception handler 268 derives three vertices including at least the fifth vertex.
  • the coordinate deriver 264 derives and outputs the fourth vertex in the three-dimensional mesh using three vertices and multiple angles.
  • the fourth vertex forms a second triangle using a common side with the triangle formed based on the three vertices.
  • a new virtual reference triangle is formed using the new vertices. . This allows vertices to be efficiently encoded or decoded.
  • ⁇ Supplementary information regarding the second aspect of encoding and decoding of vertex information> it is selected whether or not to perform exception processing based on the determination result of whether the information on the reference triangle satisfies the condition.
  • the conditions may be, for example, the following conditions.
  • Each side of the reference triangle has connectivity (see FIG. 49). For example, among the vertices of the reference triangle, a vertex located opposite the target vertex across the side shared by the triangle containing the target vertex and the reference triangle is connected to another vertex of the reference triangle. .
  • condition (i) and (ii) above are just examples, and only one of the conditions may be used for the determination, a different condition may be added, or at least one of the conditions may be used for the determination. May be replaced.
  • information on the reference triangle may be used as a condition for selection or switching.
  • a reference triangle can be defined as a triangle that shares a vertex or edge with the triangle that has the processing target vertex.
  • a triangle different from the reference triangle may be a triangle that does not share at least one of a vertex and a side with the triangle having the processing target vertex. Note that each triangle may be a polygon different from a triangle.
  • the location information of the vertices of the reference triangle, the connection information, and the information on whether the vertices are available may be used. Further, information on other triangles or information on the vertices of the triangle having the processing target apex may be used.
  • a process may be selected or switched from among a plurality of processes including the first process and the second process. That is, the plurality of processes may include processes different from the first process and the second process.
  • a reference triangle is used in the first process, and a triangle different from the reference triangle is used in the second process, but the first process and the second process are not limited to these. That is, the same triangle may be used in the first process and the second process.
  • the first process may be a process performed using a reference triangle, and the second process may be another process performed using a reference triangle.
  • the first process may be a process performed using a triangle different from the reference triangle, and the second process may be another process performed using a triangle different from the reference triangle.
  • the first process may be a process performed using a reference triangle
  • the second process may be a process performed using another triangle in which at least one side does not have connectivity.
  • a process performed using yet another triangle having connectivity on each side may be defined as the third process. Then, a process may be selected or switched from among a plurality of processes including the first process, the second process, and the third process.
  • the selectable exception handling may be different based on the determination result of the condition.
  • the combination of the condition determination result and the exception handling may be, for example, the following configuration.
  • each vertex of the reference triangle has been processed (encoded or decoded), but any side of the reference triangle does not have connectivity. , the first process, and the third process.
  • the reference triangle is used regardless of the presence or absence of connectivity, so there is a possibility that an increase in the amount of processing can be suppressed.
  • the third process there is a possibility that a triangle constituting the same object as the triangle having the target vertex is used, and the target vertex can be indicated more efficiently than the first process. It may be possible.
  • processing may be selected from among the following. In other words, processing may be performed using a triangle different from the reference triangle, regardless of connectivity. In the case of the third process, there is a possibility that the process is performed more efficiently than the second process.
  • the third process described above may be always selected.
  • the reference triangle as unavailable and use another triangle whose vertices have been processed and whose edges have connectivity. Processing may also be performed. With this configuration, processing may be performed more efficiently.
  • positional information of vertices in the present disclosure is not limited to information on coordinates of geometry, and may be other positional information different from geometry.
  • vertices that cannot be used for encoding or decoding are replaced with vertices that have already been encoded or decoded.
  • Examples of vertices that have already been encoded or decoded are the last three vertices of the encoded or decoded part of the mesh.
  • One or more additional vertices may be used to form a new reference triangle.
  • An index indicating which vertices are used to form the reference triangle may be signaled from encoding device 100 to decoding device 200. Such signaling may apply not only in exceptional cases, but also in non-exceptional cases. Also, such signaling may be applied not only to the "angle" prediction mode, but also to other prediction modes that use reference triangles, such as the parallelogram prediction mode.
  • a dummy reference triangle is generated using predefined vertices.
  • the predefined vertices may be (1,0,0), (0,1,0) and (0,0,1).
  • a new reference triangle may then be created using the predefined vertices. The new reference triangle may then be used for processing the vertices.
  • FIG. 57 is a flowchart showing an example of exception handling according to this aspect.
  • the vertices of the reference triangle are set using predefined values (S301).
  • the reference triangle is then used to encode or decode the vertices of the new triangle.
  • FIG. 58 is a flowchart showing another example of the encoding process according to this aspect.
  • the encoding device 100 derives the coordinates of the fourth vertex to be encoded from the mesh (S151).
  • the triangle having the fourth vertex is defined as the second triangle
  • the triangle that shares a side with the second triangle is defined as the reference triangle.
  • the encoding device 100 derives the connection information of the reference triangle from the mesh (S152). In the subsequent steps, the encoding device 100 determines whether the connection information of the reference triangle satisfies the condition, and executes the encoding process of the vertex coordinates based on the determination result.
  • the determination process the following two are exemplified as the determination process.
  • each side of the reference triangle has connectivity (S153). For example, whether the vertices at both ends of the side shared by the reference triangle and the second triangle are connected to the remaining vertices of the reference triangle.
  • the encoding device 100 performs the fourth Information for specifying the coordinates of the vertices is encoded (S155, S156).
  • the encoding device 100 encodes information for specifying the coordinates of the fourth vertex based on the reference triangle. become Here, an example is shown in which angle information is used for encoding (S156).
  • exception handling when (i) is not satisfied and the exception handling when (ii) is not satisfied may be different processes. Furthermore, not only connection information but also vertex information may be derived and used for determination.
  • FIG. 59 is a flowchart showing another example of the decoding process according to this aspect.
  • the decoding device 200 acquires information for specifying the coordinates of the fourth vertex to be decoded from the bitstream.
  • decoding angle information is shown (S251).
  • the triangle having the fourth vertex is defined as the second triangle, and the triangle that shares a side with the second triangle is defined as the reference triangle.
  • the decoding device 200 derives reference triangle connection information from the bitstream (S252). In the subsequent steps, the decoding device 200 determines whether the connection information of the reference triangle satisfies the condition, and executes a vertex coordinate decoding process based on the determination result.
  • the determination process the following two are exemplified as the determination process.
  • each side of the reference triangle has connectivity (S253). For example, whether the vertices at both ends of the side shared by the reference triangle and the second triangle are connected to the remaining vertices of the reference triangle.
  • the decoding device 200 determines whether the fourth vertex is The coordinates of are derived (decoded) (S255, S256). On the other hand, if both (i) and (ii) above are satisfied (Yes in S253 and Yes in S254), the decoding device 200 derives (decodes) the coordinates of the fourth vertex based on the reference triangle.
  • angle information is used for derivation (decoding) (S256).
  • exception handling when (i) is not satisfied and the exception handling when (ii) is not satisfied may be different processes. Furthermore, not only connection information but also vertex information may be derived and used for determination.
  • FIG. 60 is a conceptual diagram showing an example of the position of parameters in a bitstream.
  • Information for exception handling may be signaled as a parameter in the bitstream, as shown in FIG. 60.
  • FIG. 61 is a conceptual diagram showing another example of the position of parameters in a bitstream.
  • Information for exception handling may be signaled as a parameter in the header of the bitstream, as shown in FIG. 61.
  • the exception handling method may be indicated using parameters.
  • the parameter is an index with a value of 0 or 1. The value may indicate whether to use predefined dummy vertices or signaled dummy vertices. If signaled dummy vertices are used, the dummy vertices are also signaled after the parameters.
  • FIG. 62 is a flowchart showing an example of basic encoding processing according to this embodiment.
  • the circuit 151 of the encoding device 100 shown in FIG. 24 performs the encoding process shown in FIG. 62 in operation.
  • the circuit 151 sets a first mode set including a normal mode in which the first triangle is used to encode the encoding target vertex of the second triangle.
  • a mode for encoding the encoding target vertex is selected from among them (S162).
  • the circuit 151 provides a second triangle including an exception mode for encoding the target vertex to be encoded using a third triangle that is different from both the first triangle and the second triangle.
  • a mode is selected from the mode set (S163).
  • the circuit 151 encodes the encoding target vertex according to the mode (S164).
  • the first triangle does not satisfy the condition, it may be possible to encode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to encode the vertices of the second triangle using a reference triangle suitable for encoding.
  • the third triangle may be a triangle that shares a side with the first triangle. This may make it possible to use a third triangle in the vicinity of the first triangle used in the normal mode in the exceptional mode. Therefore, it may be possible to reduce the difference between normal mode and exception mode. Therefore, it may be possible to suppress variations in code amount.
  • the third triangle may be a triangle that shares a side with the second triangle. This may allow a third triangle in the vicinity of the second triangle to be used in the exception mode. Therefore, it may become possible to efficiently encode the encoding target vertex using the third triangle near the encoding target vertex.
  • the third triangle may be a triangle that shares an apex with the first triangle. This may make it possible to use a third triangle in the vicinity of the first triangle used in the normal mode in the exceptional mode. Therefore, it may be possible to reduce the difference between normal mode and exception mode. Therefore, it may be possible to suppress variations in code amount.
  • the third triangle may be a triangle in which each vertex has been encoded. This may make it possible to encode the encoding target vertex using the third triangle that reflects the characteristics of the three-dimensional mesh. Therefore, it may become possible to efficiently encode the encoding target vertex.
  • the third triangle may be a triangle in which each side has connectivity. This may make it possible to encode the encoding target vertex using the third triangle that reflects the characteristics of the three-dimensional mesh. Therefore, it may become possible to efficiently encode the encoding target vertex.
  • the third triangle may be a triangle having a dummy apex.
  • the third triangle may be a triangle having a dummy apex.
  • the exception mode may include multiple submodes. Then, when the exception mode is selected as the mode, the circuit 151 may select a submode for encoding the encoding target vertex from among the plurality of submodes. As a result, it may become possible to encode a target vertex to be encoded according to a submode selected from a plurality of submodes in the exception mode. Therefore, it may be possible to encode a target vertex to be encoded according to an exception handling method that is adaptively selected from among a plurality of exception handling methods.
  • the circuit 151 may encode a parameter indicating the submode. This may make it possible to share submode information for encoding the encoding target vertex between encoding device 100 and decoding device 200 according to the parameters. Therefore, it may be possible to apply the same submode for encoding and decoding according to the parameters.
  • the circuit 151 may determine the third triangle according to the submode. Thereby, by selecting a submode from among a plurality of submodes, it may be possible to select a third triangle suitable for encoding. Therefore, it may become possible to efficiently encode the encoding target vertex using the third triangle suitable for encoding.
  • the circuit 151 may determine the apex of the third triangle according to the submode. Thereby, by selecting a submode from among a plurality of submodes, it may be possible to select a vertex of the third triangle suitable for encoding. Therefore, it may become possible to efficiently encode the encoding target vertex using the third triangle suitable for encoding.
  • the condition may include that each vertex of the first triangle has been encoded.
  • each vertex of the first triangle has been encoded.
  • the condition may include that each side of the first triangle has connectivity. Thereby, it may be possible to suppress the use of a first triangle whose sides do not have connectivity in encoding the vertices of the second triangle. Then, it may be possible to use a mode suitable for encoding the vertices of the second triangle.
  • the circuit 151 may encode the angle indicating the encoding target vertex using the third triangle.
  • This may make it possible to prevent the code amount from becoming too large. That is, it may be possible to suppress the worst case code amount.
  • first triangle and the second triangle may share a side.
  • the side that the first triangle and the second triangle share may correspond to the opposite side of the encoding target vertex in the second triangle.
  • the third triangle may have a side that is shared by the first triangle and the second triangle. That is, the first triangle, the second triangle, and the third triangle may share a side. In this case, the third triangle shares a side with the first triangle, a side with the second triangle, a vertex with the first triangle, and a vertex with the second triangle.
  • the third triangle is not limited to such a triangle, and may have any side or any vertex of the first triangle, for example.
  • the normal mode may be a mode in which the encoding target vertex is encoded using the first triangle and one or more angles.
  • the exception mode may be a mode in which a vertex to be encoded is encoded using a third triangle and one or more angles.
  • the first mode set and the second mode set are different from each other.
  • the normal mode may not be included in the second mode set, but may be included only in the first mode set.
  • the exception mode may not be included in the first mode set, but may be included only in the second mode set.
  • the first mode set includes a mode in which a vertex to be encoded is encoded using coordinates, a mode in which a vertex to be encoded is encoded using a parallelogram, and a mode in which a vertex to be encoded is encoded using a polynomial.
  • the parallelogram may be based on the first triangle.
  • the normal mode may be a mode in which the encoding target vertex is encoded using a parallelogram based on the first triangle.
  • the second mode set includes a mode for encoding the encoding target vertex using coordinates, a mode for encoding the encoding target vertex using a parallelogram, and a mode for encoding the encoding target vertex using a polynomial.
  • the parallelogram may be based on the third triangle.
  • the exception mode may be a mode in which the encoding target vertex is encoded using a parallelogram based on the third triangle.
  • the first mode set may include only the normal mode. Then, the circuit 151 may select the normal mode if the first triangle satisfies the condition. That is, if the first triangle satisfies the condition, the circuit 151 may encode the encoding target vertex according to the normal mode.
  • the second mode set may include only the exception mode.
  • the circuit 151 may then select an exception mode if the first triangle does not satisfy the condition. That is, if the first triangle does not satisfy the condition, the circuit 151 may encode the encoding target vertex according to the exception mode.
  • the circuit 151 may select a mode from a mode set including a normal mode and an exception mode.
  • the mode set may correspond to both the first mode set and the second mode set. Then, if the first triangle satisfies the condition, the circuit 151 may select the normal mode and encode the encoding target vertex according to the normal mode. Furthermore, if the first triangle does not satisfy the condition, the circuit 151 may select an exception mode and encode the encoding target vertex according to the exception mode.
  • the plurality of submodes may include a submode in which the encoding target vertex is encoded using a third triangle having a dummy vertex.
  • the plurality of sub-modes may include a sub-mode in which vertices to be encoded are encoded using a third triangle whose vertices have already been encoded.
  • the plurality of sub-modes may include a sub-mode in which the encoding target vertex is encoded using a third triangle, each side of which has connectivity.
  • the submode may correspond to the condition of the third triangle, or may correspond to the condition of the apex of the third triangle.
  • the submode may correspond to the third triangle or the method for selecting the apex of the third triangle.
  • the normal mode may include multiple submodes. Then, when the normal mode is selected as the mode, the circuit 151 may select a submode for encoding the encoding target vertex from among the plurality of submodes. The circuit 151 may determine the first triangle or the vertex of the first triangle according to the submode. In other words, the submode may correspond to a method of selecting the first triangle or the apex of the first triangle.
  • a submode included in the exception mode may be expressed as an exception submode.
  • a submode included in the normal mode may be expressed as a normal submode.
  • a mode may be expressed as a process.
  • the normal mode, exception mode, and submode may also be expressed as normal processing, exception processing, and subprocessing.
  • the circuit 151 may encode the encoding target vertex by entropy encoding. Specifically, the circuit 151 may encode an angle indicating a vertex to be encoded by entropy encoding. Also, for example, the circuit 151 may encode one, two, or three angles for identifying the vertex to be encoded.
  • the first triangle may be a reference triangle that is determined as a triangle close to the encoding target vertex according to the reference rule.
  • each of the vertices described above may be expressed by geometric coordinates in a three-dimensional mesh.
  • FIG. 63 is a flowchart illustrating an example of basic decoding processing according to this embodiment.
  • the circuit 251 of the decoding device 200 shown in FIG. 25 performs the decoding process shown in FIG. 63 in operation.
  • the circuit 251 selects one of the first mode sets from the first mode set including the normal mode in which the first triangle is used to decode the decoding target vertex of the second triangle. A mode for decoding a vertex to be decoded is selected (S262). On the other hand, if the first triangle does not satisfy the condition (No in S261), the circuit 251 sets a second mode set including an exception mode in which the vertex to be decoded is decoded using a third triangle that is different from both the first triangle and the second triangle. Select a mode from among them (S263).
  • the circuit 251 decodes the decoding target vertex according to the mode (S264).
  • the first triangle does not satisfy the condition, it may become possible to decode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to decode the vertices of the second triangle using a reference triangle suitable for decoding.
  • the third triangle may be a triangle that shares a side with the first triangle. This may make it possible to use a third triangle in the vicinity of the first triangle used in the normal mode in the exceptional mode. Therefore, it may be possible to reduce the difference between normal mode and exception mode. Therefore, it may be possible to suppress variations in code amount.
  • the third triangle may be a triangle that shares a side with the second triangle. This may allow a third triangle in the vicinity of the second triangle to be used in the exception mode. Therefore, it may be possible to efficiently decode the decoding target vertex using the third triangle near the decoding target vertex.
  • the third triangle may be a triangle that shares an apex with the first triangle. This may make it possible to use a third triangle in the vicinity of the first triangle used in the normal mode in the exceptional mode. Therefore, it may be possible to reduce the difference between normal mode and exception mode. Therefore, it may be possible to suppress variations in code amount.
  • the third triangle may be a triangle in which each vertex has been decoded. This may make it possible to decode the decoding target vertex using the third triangle that reflects the characteristics of the three-dimensional mesh. Therefore, it may become possible to efficiently decode the decoding target vertex.
  • the third triangle may be a triangle in which each side has connectivity. This may make it possible to decode the decoding target vertex using the third triangle that reflects the characteristics of the three-dimensional mesh. Therefore, it may become possible to efficiently decode the decoding target vertex.
  • the third triangle may be a triangle having a dummy apex. This may make it possible to decode the decoding target vertex using the third triangle having an arbitrary vertex. Therefore, it may become possible to flexibly decode the decoding target vertex.
  • the exception mode may include multiple submodes. Then, when the exception mode is selected as the mode, the circuit 251 may select a submode for decoding the decoding target vertex from among the plurality of submodes. This may make it possible to decode a decoding target vertex according to a submode selected from a plurality of submodes in the exception mode. Therefore, it may be possible to decode a decoding target vertex according to an exception handling method that is adaptively selected from a plurality of exception handling methods.
  • the circuit 251 may decode a parameter indicating a submode. This may make it possible to share submode information for decoding a decoding target vertex between encoding device 100 and decoding device 200 according to the parameters. Therefore, it may be possible to apply the same submode for encoding and decoding according to the parameters.
  • the circuit 251 may determine the third triangle according to the submode. Thereby, by selecting a submode from among a plurality of submodes, it may be possible to select a third triangle suitable for decoding. Therefore, it may become possible to efficiently decode the decoding target vertex using the third triangle suitable for decoding.
  • the circuit 251 may determine the apex of the third triangle according to the submode. Thereby, by selecting a submode from among a plurality of submodes, it may be possible to select a vertex of the third triangle suitable for decoding. Therefore, it may become possible to efficiently decode the decoding target vertex using the third triangle suitable for decoding.
  • the condition may include that each vertex of the first triangle has been decoded.
  • each vertex of the first triangle has been decoded.
  • the condition may include that each side of the first triangle has connectivity. Thereby, it may be possible to suppress the use of a first triangle whose sides do not have connectivity in decoding the vertices of the second triangle. Then, it may become possible to use a mode suitable for decoding the vertex of the second triangle.
  • the circuit 251 may decode the angle indicating the decoding target vertex using the third triangle. This may make it possible to prevent the code amount from becoming too large. That is, it may be possible to suppress the worst case code amount.
  • first triangle and the second triangle may share a side.
  • the side shared by the first triangle and the second triangle may correspond to the opposite side of the decoding target vertex in the second triangle.
  • the third triangle may have a side that is shared by the first triangle and the second triangle. That is, the first triangle, the second triangle, and the third triangle may share a side. In this case, the third triangle shares a side with the first triangle, a side with the second triangle, a vertex with the first triangle, and a vertex with the second triangle.
  • the third triangle is not limited to such a triangle, and may have any side or any vertex of the first triangle, for example.
  • the normal mode may be a mode in which the decoding target vertex is decoded using the first triangle and one or more angles.
  • the exception mode may be a mode in which the target vertex is decoded using a third triangle and one or more angles.
  • the first mode set and the second mode set are different from each other.
  • the normal mode may not be included in the second mode set, but may be included only in the first mode set.
  • the exception mode may not be included in the first mode set, but may be included only in the second mode set.
  • the first mode set includes a mode for decoding the target vertex to be decoded using coordinates, a mode for decoding the target vertex for decoding using a parallelogram, and a mode for decoding the target vertex for decoding using a polynomial. It may contain at least one of these.
  • the parallelogram may be based on the first triangle.
  • the normal mode may be a mode in which the decoding target vertex is decoded using a parallelogram based on the first triangle.
  • the second mode set includes a mode in which a vertex to be decoded is decoded using coordinates, a mode in which a vertex to be decoded is decoded using a parallelogram, and a mode in which a vertex to be decoded is decoded using a polynomial. It may contain at least one of these.
  • the parallelogram may be based on the third triangle.
  • the exception mode may be a mode in which the decoding target vertex is decoded using a parallelogram based on the third triangle.
  • the first mode set may include only the normal mode. Then, the circuit 251 may select the normal mode if the first triangle satisfies the condition. That is, if the first triangle satisfies the condition, the circuit 251 may decode the decoding target vertex according to the normal mode.
  • the second mode set may include only the exception mode.
  • the circuit 251 may then select an exception mode if the first triangle does not satisfy the condition. That is, if the first triangle does not satisfy the condition, the circuit 251 may decode the decoding target vertex according to the exception mode.
  • the circuit 251 may select a mode from a mode set including a normal mode and an exception mode.
  • the mode set may correspond to both the first mode set and the second mode set. Then, if the first triangle satisfies the condition, the circuit 251 may select the normal mode and decode the decoding target vertex according to the normal mode. Further, if the first triangle does not satisfy the condition, the circuit 251 may select an exception mode and decode the decoding target vertex according to the exception mode.
  • the plurality of submodes may include a submode in which the decoding target vertex is decoded using a third triangle having a dummy vertex.
  • the plurality of submodes may include a submode in which the decoding target vertex is decoded using a third triangle whose vertices have already been decoded.
  • the plurality of sub-modes may include a sub-mode in which the decoding target vertex is decoded using a third triangle whose sides have connectivity.
  • the submode may correspond to the condition of the third triangle, or may correspond to the condition of the apex of the third triangle.
  • the submode may correspond to the third triangle or the method for selecting the apex of the third triangle.
  • the normal mode may include multiple submodes. Then, when the normal mode is selected as the mode, the circuit 251 may select a submode for decoding the decoding target vertex from among the plurality of submodes. Circuit 251 may determine the first triangle or the vertex of the first triangle according to the submode. In other words, the submode may correspond to a method of selecting the first triangle or the apex of the first triangle.
  • a submode included in the exception mode may be expressed as an exception submode.
  • a submode included in the normal mode may be expressed as a normal submode.
  • a mode may be expressed as a process.
  • the normal mode, exception mode, and submode may also be expressed as normal processing, exception processing, and subprocessing.
  • the circuit 251 may decode the decoding target vertex by entropy decoding. Specifically, the circuit 251 may decode the angle indicating the decoding target vertex by entropy decoding. Also, for example, the circuit 251 may decode one, two, or three angles for identifying a vertex to be decoded.
  • the first triangle may be a reference triangle that is determined as a triangle close to the decoding target vertex according to the reference rule.
  • each of the vertices described above may be expressed by geometric coordinates in a three-dimensional mesh.
  • FIG. 64 is a block diagram showing yet another configuration example of the encoding device 100 according to the present embodiment.
  • the encoding device 100 includes a selector 171 and an encoding processor 172.
  • the selector 171 is, for example, an electric circuit.
  • the selector 171 may correspond to the above-described traverser 161, determiner 167, exception handler 168, etc., or may be implemented with the above-described circuit 151 and memory 152.
  • the encoding processor 172 is, for example, an electric circuit.
  • the encoding processor 172 may correspond to the angle deriving device 163, entropy encoder 166, etc. described above, or may be implemented with the circuit 151 and memory 152 described above.
  • the selector 171 selects a mode from the first mode set including the normal mode, and if the first triangle does not satisfy the condition, the selector 171 selects the mode from the second mode set including the exception mode. Select a mode from. Then, the encoding processor 172 encodes the encoding target vertices according to the mode.
  • the first triangle does not satisfy the condition, it may be possible to encode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to encode the vertices of the second triangle using a reference triangle suitable for encoding.
  • FIG. 65 is a block diagram showing yet another configuration example of the decoding device 200 according to the present embodiment.
  • the decoding device 200 includes a selector 271 and a decoding processor 272.
  • the selector 271 is, for example, an electric circuit.
  • the selector 271 may correspond to the traverser 261, determiner 267, exception handler 268, etc. described above, or may be implemented with the circuit 251 and memory 252 described above.
  • the decoding processor 272 is, for example, an electric circuit.
  • the decoding processor 272 may correspond to the coordinate deriving device 264, entropy decoder 266, etc. described above, or may be implemented with the circuit 251 and memory 252 described above.
  • the selector 271 selects a mode from the first mode set including the normal mode, and if the first triangle does not satisfy the condition, the selector 271 selects a mode from the second mode set including the exception mode. Select a mode from. Then, the decoding processor 272 decodes the decoding target vertex according to the mode.
  • the first triangle does not satisfy the condition, it may become possible to decode the vertex of the second triangle using a third triangle different from the first triangle. Therefore, it may be possible to decode the vertices of the second triangle using a reference triangle suitable for decoding.
  • a process executed by a specific component in the embodiment may be executed by another component instead of the specific component.
  • the order of the plurality of processes may be changed, or the plurality of processes may be executed in parallel.
  • the encoding and decoding of the present disclosure can be applied to encoding and decoding of vertex information indicating the position of a vertex.
  • the encoding and decoding of the present disclosure is not limited to the vertices of a surface in a three-dimensional mesh, and may be applied to encoding and decoding of vertex information indicating the positions of other vertices.
  • each process of the present disclosure may be performed as one of a plurality of selectable processes.
  • At least some of the configurations of the present disclosure may be implemented as an integrated circuit. At least some of the multiple processes of the present disclosure may be used as an encoding method or a decoding method.
  • a program for causing a computer to execute the encoding method or the decoding method may be used.
  • a non-temporary computer-readable recording medium may be used in which the program is recorded.
  • a bitstream for causing the decoding device 200 to perform decoding processing may be used.
  • At least a portion of the multiple configurations and multiple processes of the present disclosure may be used as a transmitting device, a receiving device, a transmitting method, and a receiving method.
  • a program for causing a computer to execute the transmitting method or the receiving method may be used.
  • a non-temporary computer-readable recording medium may be used in which the program is recorded.
  • the present disclosure is useful, for example, for encoding devices, decoding devices, transmitting devices, receiving devices, etc. related to three-dimensional meshes, and is applicable to computer graphics systems, three-dimensional data display systems, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/JP2023/021028 2022-06-06 2023-06-06 符号化装置、復号装置、符号化方法及び復号方法 Ceased WO2023238867A1 (ja)

Priority Applications (7)

Application Number Priority Date Filing Date Title
JP2024526458A JP7723199B2 (ja) 2022-06-06 2023-06-06 符号化装置、復号装置、符号化方法及び復号方法
EP23819847.7A EP4538977A4 (en) 2022-06-06 2023-06-06 ENCODING DEVICE, DECODING DEVICE, ENCODING METHOD AND DECODING METHOD
CN202380043575.9A CN119301642A (zh) 2022-06-06 2023-06-06 编码装置、解码装置、编码方法及解码方法
KR1020247038325A KR20250019630A (ko) 2022-06-06 2023-06-06 부호화 장치, 복호 장치, 부호화 방법 및 복호 방법
MX2024014355A MX2024014355A (es) 2022-06-06 2024-11-20 Dispositivo de codificacion, dispositivo de decodificacion, metodo de codificacion y metodo de decodificacion
US18/965,181 US20250095214A1 (en) 2022-06-06 2024-12-02 Encoding device, decoding device, encoding method, and decoding method
JP2025128166A JP2025166019A (ja) 2022-06-06 2025-07-31 符号化装置、復号装置、送信装置、符号化方法、復号方法及び送信方法

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202263349243P 2022-06-06 2022-06-06
US63/349,243 2022-06-06
US202363465033P 2023-05-09 2023-05-09
US63/465,033 2023-05-09

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/965,181 Continuation US20250095214A1 (en) 2022-06-06 2024-12-02 Encoding device, decoding device, encoding method, and decoding method

Publications (1)

Publication Number Publication Date
WO2023238867A1 true WO2023238867A1 (ja) 2023-12-14

Family

ID=89118370

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/021028 Ceased WO2023238867A1 (ja) 2022-06-06 2023-06-06 符号化装置、復号装置、符号化方法及び復号方法

Country Status (7)

Country Link
US (1) US20250095214A1 (https=)
EP (1) EP4538977A4 (https=)
JP (2) JP7723199B2 (https=)
KR (1) KR20250019630A (https=)
CN (1) CN119301642A (https=)
MX (1) MX2024014355A (https=)
WO (1) WO2023238867A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025148072A1 (zh) * 2024-01-12 2025-07-17 Oppo广东移动通信有限公司 编解码方法、码流、编码器、解码器以及存储介质
WO2025182831A1 (ja) * 2024-02-28 2025-09-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化方法、復号方法、符号化装置及び復号装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20260087740A1 (en) * 2024-09-24 2026-03-26 Roblox Corporation Merging coplanar convex polygons in constructive solid geometry (csg)
CN121120809B (zh) * 2025-11-12 2026-02-10 摩尔线程智能科技(北京)股份有限公司 图元编码方法、装置、设备、存储介质及程序产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000224582A (ja) * 1998-10-02 2000-08-11 Samsung Electronics Co Ltd 漸進的な三次元メッシュ情報の符号化／復号化方法及びその装置
WO2010089380A1 (en) * 2009-02-06 2010-08-12 Thomson Licensing Method and apparatus for encoding 3d mesh models, and method and apparatus for decoding encoded 3d mesh models
US8830235B1 (en) * 1999-09-13 2014-09-09 Alcatel Lucent Non-uniform relaxation procedure for multiresolution mesh processing
WO2023074673A1 (ja) * 2021-10-26 2023-05-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、復号装置、符号化方法及び復号方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6577310B1 (en) * 1998-12-01 2003-06-10 Samsung Electronics Co., Ltd. 3D mesh coding/decoding method and apparatus for error resilience and incremental rendering
GB2561824B (en) * 2017-04-19 2020-04-01 Canon Kk Encoding and decoding of geometry data in a 3D model based on evaluated regularity of the model
US10783669B2 (en) * 2018-09-26 2020-09-22 Google Llc Texture coordinate compression using texture atlas

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000224582A (ja) * 1998-10-02 2000-08-11 Samsung Electronics Co Ltd 漸進的な三次元メッシュ情報の符号化／復号化方法及びその装置
JP2006187015A (ja) 1998-10-02 2006-07-13 Samsung Electronics Co Ltd 漸進的な三次元メッシュ情報の符号化／復号化方法及びその装置
US8830235B1 (en) * 1999-09-13 2014-09-09 Alcatel Lucent Non-uniform relaxation procedure for multiresolution mesh processing
WO2010089380A1 (en) * 2009-02-06 2010-08-12 Thomson Licensing Method and apparatus for encoding 3d mesh models, and method and apparatus for decoding encoded 3d mesh models
WO2023074673A1 (ja) * 2021-10-26 2023-05-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、復号装置、符号化方法及び復号方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MARTIN ISENBURG ; PIERRE ALLIEZ: "Compressing polygon mesh geometry with parallelogram prediction", VIS 2002. IEEE VISUALIZATION 2002. PROCEEDINGS. BOSTON, MA, OCT. 27 - NOV. 1, 2002., NEW YORK, NY : IEEE., US, 27 October 2002 (2002-10-27) - 1 November 2002 (2002-11-01), US , pages 141 - 146, XP058097048, ISBN: 978-0-7803-7498-0 *
See also references of EP4538977A4
ZHONGBAO CHEN ; FULIANG BAO ; ZHIGANG FANG ; ZHEN LI: "Compression of 3D Triangle Meshes with a Generalized Parallelogram Prediction Scheme Based on Vector Quantization", INTELLIGENT INFORMATION HIDING AND MULTIMEDIA SIGNAL PROCESSING (IIH-MSP), 2010 SIXTH INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 15 October 2010 (2010-10-15), Piscataway, NJ, USA , pages 184 - 187, XP031803533, ISBN: 978-1-4244-8378-5 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025148072A1 (zh) * 2024-01-12 2025-07-17 Oppo广东移动通信有限公司 编解码方法、码流、编码器、解码器以及存储介质
WO2025182831A1 (ja) * 2024-02-28 2025-09-04 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化方法、復号方法、符号化装置及び復号装置

Also Published As

Publication number Publication date
JP7723199B2 (ja) 2025-08-13
JPWO2023238867A1 (https=) 2023-12-14
JP2025166019A (ja) 2025-11-05
MX2024014355A (es) 2024-12-06
KR20250019630A (ko) 2025-02-10
EP4538977A4 (en) 2025-10-08
CN119301642A (zh) 2025-01-10
EP4538977A1 (en) 2025-04-16
US20250095214A1 (en) 2025-03-20

Similar Documents

Publication Publication Date Title
WO2023074673A1 (ja) 符号化装置、復号装置、符号化方法及び復号方法
JP7723199B2 (ja) 符号化装置、復号装置、符号化方法及び復号方法
WO2024075608A1 (ja) 符号化装置、復号装置、符号化方法及び復号方法
US20250322550A1 (en) Encoding method, decoding method, encoding device, and decoding device
US20260004466A1 (en) Encoding device, decoding device, encoding method, and decoding method
US20260057559A1 (en) Decoding method, encoding method, decoding device, and encoding device
US20250371743A1 (en) Encoding method, decoding method, encoding device, and decoding device
US20240135594A1 (en) Adaptive geometry filtering for mesh compression
US20260120332A1 (en) Encoding device, decoding device, encoding method, and decoding method
CN118140250A (zh) 编码装置、解码装置、编码方法及解码方法
WO2025004935A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2025130352A1 (en) Dynamic mesh base geometry position prediction coding
WO2025123959A1 (en) Dynamic mesh base geometry position prediction coding
WO2025070342A1 (ja) 符号化装置、復号装置、符号化方法及び復号方法
WO2025009527A1 (ja) 符号化装置、復号装置、符号化方法及び復号方法
WO2025216315A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2025220615A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2025150425A1 (ja) 復号方法、符号化方法、復号装置及び符号化装置
WO2026084019A1 (ja) 復号方法、符号化方法、復号装置及び符号化装置
WO2025220701A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2025220696A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2026009768A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
WO2026094612A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置
TW202548673A (zh) 編碼方法、解碼方法、編碼裝置及解碼裝置
WO2025216144A1 (ja) 符号化方法、復号方法、符号化装置及び復号装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23819847

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: MX/A/2024/014355

Country of ref document: MX

WWE Wipo information: entry into national phase

Ref document number: 202447091728

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2024526458

Country of ref document: JP

Ref document number: 202380043575.9

Country of ref document: CN

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112024024186

Country of ref document: BR

WWE Wipo information: entry into national phase

Ref document number: 2023819847

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 202380043575.9

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2023819847

Country of ref document: EP

Effective date: 20250107

WWP Wipo information: published in national office

Ref document number: 1020247038325

Country of ref document: KR

REG Reference to national code

Ref country code: BR

Ref legal event code: B01E

Ref document number: 112024024186

Country of ref document: BR

Free format text: 1) APRESENTE NOVO RELATORIO DESCRITIVO ADAPTADO AO ART. 26 INCISO I DA PORTARIA/INPI NO 14/2024, UMA VEZ QUE O CONTEUDO ENVIADO NA PETICAO NO 870240099025 DE 21/11/2024 ENCONTRA-SE FORA DA NORMA EM RELACAO AO TITULO, CONTENDO TEXTO DIFERENTE DO TITULO INCIANDO A PAGINA. 2) APRESENTE NOVO RESUMO ADAPTADO AO ART. 40 INCISO I DA PORTARIA/INPI NO 14/2024, UMA VEZ QUE O CONTEUDO ENVIADO NA PETICAO NO 870240099025 DE 21/11/2024 ENCONTRA-SE FORA DA NORMA EM RELACAO AO TITULO, CONTENDO TEXTO DIFERENTE DO TITULO INCIANDO A PAGINA. A EXIGENCIA DEVE SER RESPONDIDA EM ATE 60 (SESSENTA) DIAS DE SUA PUBLICACAO E DEVE SER REALIZADA POR MEIO DA PETICAO GRU CODIGO DE SERVICO 207.

WWP Wipo information: published in national office

Ref document number: 2023819847

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 112024024186

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20241121