CN111866484B

CN111866484B - Point cloud encoding method, point cloud decoding method, device and storage medium

Info

Publication number: CN111866484B
Application number: CN201910364032.2A
Authority: CN
Inventors: 蔡康颖; 张德军
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2023-06-20
Anticipated expiration: 2039-04-30
Also published as: WO2020220941A1; CN111866484A

Abstract

The application discloses a point cloud encoding method, a point cloud decoding device and a storage medium, and belongs to the field of data processing. The method comprises the following steps: auxiliary information of a patch to be encoded and a first identification of the patch to be encoded are acquired. Because the number of the first type of patch in the current point cloud group in each point cloud frame can be the same, when the first identifier is a first value and the patch to be encoded is the first patch in the current point cloud frame, the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream. When the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, auxiliary information of the to-be-encoded patch can be encoded into the code stream without encoding the first identifier of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified, the bit cost of the code stream can be further reduced, and the encoding efficiency can be improved.

Description

Point cloud encoding method, point cloud decoding method, device and storage medium

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a point cloud encoding method, a point cloud decoding method, a device, and a storage medium.

Background

Along with the continuous development of the three-dimensional sensing technology, the acquisition of the point cloud is more and more convenient, the quality of the acquired point cloud is higher, and the scale is larger. Therefore, how to effectively encode and decode the point cloud is a current urgent problem to be solved.

Disclosure of Invention

The application provides a point cloud encoding method, a point cloud decoding device and a storage medium, which can solve the problem of how to effectively encode and decode point clouds in the related technology. The technical scheme is as follows:

in a first aspect, a point cloud encoding method is provided, the method including: acquiring auxiliary information of a patch to be encoded (point cloud block) and a first identification of the patch to be encoded; when the first mark is a first value, the auxiliary information of the patch to be encoded and the first mark are encoded into a code stream, and the patch to be encoded is the first patch in the current point cloud frame; or when the first mark is a first value, encoding auxiliary information of the patch to be encoded into a code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame; the first mark is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded comprises three-dimensional information.

It should be noted that the patch to be encoded may be any one of patches included in the current point cloud frame to be encoded. The first identification may indicate the type of patch to be encoded. The syntax element of the first flag may be a patch_mode or a patch_type, etc. The first identifier has different values, and the type of the patch to be encoded is different.

When the first identifier is a first value, the to-be-encoded patch may be indicated to be a first type patch, where the first type patch may be a global matching patch in the current point cloud group, and the auxiliary information of the to-be-encoded patch includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information.

When the first type of patch exists in the current point cloud group, the quantity of the first type of patch in each point cloud frame in the current point cloud group can be the same, and when the quantity of the first type of patch is larger than 1, the first type of patch in each point cloud frame in the current point cloud group can be sequentially arranged according to the same sequence, so when the first mark is a first value, the patch to be coded is the first patch in the current point cloud frame, the first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information and the first mark of the patch to be coded can be coded into the code stream; when the first mark is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the to-be-encoded patch is indicated to be a non-first type patch in the current point cloud frame, and at the moment, auxiliary information of the to-be-encoded patch can be encoded into a code stream without encoding the first mark of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified.

Because the first class of patches have the patches with the matching relationship in all the point cloud frames in the current point cloud group, the two-dimensional information, the rotation transformation information and the normal coordinate axis information among the first class of patches in the current point cloud group are the same, and the three-dimensional information may not be the same. Therefore, in order to reduce the bit overhead of the code stream and further improve the coding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be coded may include three-dimensional information, but not include two-dimensional information, rotation transformation information and normal coordinate axis information.

In one possible scenario, the patches in each point cloud frame may be arranged in the order of the first type of patch, the second type of patch, and the third type of patch, and then encoded in the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, if the first type of patch exists in the current point cloud group, the first patch in each point cloud frame is the first type of patch. The number of the first class of patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first class of patches is greater than 1, the first class of patches in each point cloud frame in the current point cloud group can be sequentially arranged according to the same sequence, that is, the first class of patches in the same position in each point cloud frame have a matching relationship.

However, in order to facilitate distinguishing which point cloud frames can form a point cloud group, as shown in fig. 10, for the first point cloud frame in the current point cloud group, the first identifier of all the patches included in the first point cloud frame may be set to the third value, and the patches in other point cloud frames are arranged according to the order of the first type of patch, the second type of patch and the third type of patch. Thus, when the first identifier of the patch to be encoded is a first value, it may be determined that the current point cloud frame is a non-first point cloud frame within the current point cloud group. Moreover, the current point cloud frame may be a kth point cloud frame in the current point cloud group, where K is an integer greater than or equal to 3. That is, for the 3 rd point cloud frame and each point cloud frame after the 3 rd point cloud frame in the current point cloud group, the encoding may be performed in the manner of S902.

The point cloud frames in the current point cloud group can be numbered according to two numbering modes. First numbering scheme: point cloud frame 0, point cloud frame 1, point cloud frame 2 … …; the second numbering scheme: point cloud frame 1, point cloud frame 2, point cloud frame 3 … …. The kth point cloud frame may refer to the point cloud frame 2 and any point cloud frame after the point cloud frame 2 in the first numbering mode, or may refer to the point cloud frame 3 and any point cloud frame after the point cloud frame 3 in the second numbering mode. Regardless of the numbering mode of the point cloud frames in the current point cloud group, the kth point cloud frame can refer to the 3 rd point cloud frame and the 3 rd and subsequent point cloud frames in the actual current point cloud group.

Based on the above description, the 3 rd point cloud frame and each point cloud frame after the 3 rd point cloud frame in the current point cloud group may be encoded in the above manner, and the 2 nd point cloud frame in the current point cloud group may be encoded in the following manner:

in a first possible implementation manner, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, when the first identifier of the patch to be encoded is the first value and the patch to be encoded is the first patch in the current point cloud frame, the first identifier of the patch to be encoded, the number of the first class patches in the current point cloud group, and the auxiliary information of the patch to be encoded may be encoded into the code stream. When the first identifier of the patch to be encoded is a first value and the patch to be encoded is a non-first patch in the current point cloud frame, the auxiliary information of the patch to be encoded can be encoded into the code stream without encoding the first identifier of the patch to be encoded into the code stream.

In the process of setting the first identifiers of all the patches included in the first point cloud frame in the current point cloud group to the third value, the number of the first type patches in the current point cloud group can be obtained.

Moreover, the operation of encoding the first identifier of the patch to be encoded, the number of the first class patches in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: encoding a first identification of a patch to be encoded into a code stream, encoding the number of the first type of patch in the current point cloud group into the code stream, and finally encoding auxiliary information of the patch to be encoded into the code stream. That is, the number of the first type of patch in the current point cloud group is encoded into the code stream after the first identification of the patch to be encoded.

It should be noted that, the number of the first type of latches may be encoded into the code stream by using a fixed-length encoding method, or the number of the first type of latches may be encoded into the code stream by using a variable-length encoding method, which is not limited in the embodiment of the present application.

In a second possible implementation manner, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, when the first identifier of the patch to be encoded is the first value, the first identifier of the patch to be encoded and the auxiliary information may be encoded into the code stream. That is, the first identification and the auxiliary information of all the first type patches in the 2 nd point cloud frame need to be encoded into the code stream, so that the number of the first type patches in the current point cloud group can be indicated, and the number of the first type patches in the current point cloud group does not need to be encoded into the code stream.

No matter what implementation mode is used for encoding the first type of patch in the 2 nd point cloud frame, the number of the first type of patch in the current point cloud group can be determined through the 2 nd point cloud frame. Thus, for the 3 rd point cloud frame and the point cloud frames after the 3 rd point cloud frame in the current point cloud group, only the first mark of the first patch, namely the first type patch, can be encoded into the code stream, and the first mark of the first type patch is not encoded into the code stream.

In summary, when the first identifier of the patch to be encoded is the third value and the patch to be encoded is the first patch in the current point cloud frame, the current point cloud frame is determined to be the first point cloud frame in the current point cloud group. Thus, the length of the point cloud frame of the current point cloud group does not need to be encoded in the code stream. And, can only encode the first label of the first type patch into the code stream to the 3 rd point cloud frame and point cloud frame after 3 rd point cloud frame in the current point cloud group, and does not need to encode the first label of the non-first type patch into the code stream, have raised the coding efficiency, have reduced the bit overhead of the code stream at the same time.

It should be noted that, in one possible implementation, the first patch in the current point cloud frame may correspond to an identification information, which may also be referred to as an index, and through which the first patch in the current point cloud frame may be uniquely indicated, and the identification information of the first patch may be generally encoded into the code stream. For other patches in the current point cloud frame, a counter may be used to determine what the other patches are in the current point cloud frame. Wherein the counter starts counting when determining the first patch of the current point cloud frame.

It should be noted that the three-dimensional information of the patch to be encoded may include an offset (3d_shift_agent_axis) of the patch to be encoded in the 3D space along the tangential direction, an offset (3d_shift_agent_axis) of the patch to be encoded in the 3D space along the double tangential direction, and an offset (3d_shift_normal_axis) of the patch to be encoded in the 3D space along the normal direction.

As an example, when the first identifier is the first value, the three-dimensional information of the to-be-encoded latch may be encoded into the code stream according to the three-dimensional information of the reference latch of the to-be-encoded latch, specifically, the three-dimensional information of the to-be-encoded latch may be encoded into the code stream by adopting a differential encoding mode, or may be directly encoded into the code stream, and the specific encoding mode is not limited. And encoding the first identification of the patch to be encoded into the code stream by adopting a fixed-length encoding mode.

The step of encoding the three-dimensional information of the patch to be encoded into the code stream by adopting a differential encoding mode refers to encoding a difference value between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.

And when the first identifier is a second value or a third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream.

When the first identifier is a second value, the to-be-encoded patch is indicated to be a second type patch, the second type patch can be a local matching patch in the current point cloud group, and the auxiliary information of the to-be-encoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-encoded patch. When the first mark is a third value, the to-be-encoded patch is indicated to be a third type patch, the third type patch can be a non-matching patch in the current point cloud group, and the auxiliary information of the to-be-encoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Because the second class of patches has the patches with the matching relationship in the reference frame of the current point cloud frame, the rotation transformation information and the normal coordinate axis information between the second class of patches and the reference patches thereof are the same, and the two-dimensional information and the three-dimensional information are possibly different, and the third class of patches does not have the patches with the matching relationship in the reference frame of the current point cloud frame, in order to reduce the bit cost of the code stream and further improve the coding efficiency, when the first identifier is a first value, the auxiliary information of the patch to be coded can comprise the three-dimensional information, but does not comprise the two-dimensional information, the rotation transformation information and the normal coordinate axis information. When the first flag is a second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of a reference patch of the patch to be encoded, without including rotation transformation information and normal coordinate axis information. When the first flag is a third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that, the two-dimensional information of the to-be-encoded latch may include an offset (2d_shift_u) of the to-be-encoded latch along the u-axis direction in the current point cloud frame occupancy map, an offset (2d_shift_v) of the to-be-encoded latch along the v-axis direction in the current point cloud frame occupancy map, a width (2d_size_u) of the to-be-encoded latch in the current point cloud frame occupancy map, and a height (2d_size_v) of the to-be-encoded latch in the current point cloud frame occupancy map. In addition, the syntax element of the reference patch of the patch to be encoded may be a patch index for indicating identification information of the reference patch; the syntax element may further include identification information of a reference frame in which the reference patch is located: frameIndex. The syntax element of the rotation transform information of the patch to be encoded may be a rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded. The normal axis information may be a normal axis index of the patch to be encoded, e.g., normal axis index may be 0,1, or 2 for the X-axis, Y-axis, and Z-axis, respectively.

As an example, when the first identifier is the second value, the two-dimensional information and the three-dimensional information of the patch to be encoded may be encoded into the code stream according to the two-dimensional information and the three-dimensional information of the reference patch of the patch to be encoded, specifically, the two-dimensional information and the three-dimensional information of the patch to be encoded may be encoded into the code stream by adopting a differential encoding mode, or the two-dimensional information and the three-dimensional information of the patch to be encoded may be encoded into the code stream by adopting a more accurate prediction differential mode, and the specific encoding mode is not limited. And encoding the identification information of the reference patch of the patch to be encoded and the first identification into the code stream by adopting a fixed-length encoding mode. When the first identifier is the third value, the two-dimensional information, the three-dimensional information, the rotation transformation information, the normal coordinate axis information and the first identifier of the patch to be encoded may be encoded into the code stream by adopting a fixed-length encoding mode, or the two-dimensional information, the three-dimensional information, the rotation transformation information, the normal coordinate axis information and the first identifier of the patch to be encoded may be encoded into the code stream by adopting a variable-length encoding mode.

The encoding of the two-dimensional information and the three-dimensional information of the patch to be encoded into the code stream by adopting the differential encoding mode refers to encoding the difference value between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be encoded and the difference value between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.

Based on the above description, for the current point cloud frame, the first identifier of the first class of patch in the current point cloud frame, the first identifier of each second class of patch, and the first identifier of each third class of patch need to be encoded into the code stream. The auxiliary information of each patch included in the current point cloud frame also needs to be encoded into the code stream. There are three possible implementations of the encoding of the first identification and the auxiliary information.

In a first possible implementation manner, first identifiers of a plurality of latches included in the current point cloud frame may be packaged together and encoded into a code stream, where the first identifiers of the plurality of latches include first identifiers of latches to be encoded, and the remaining first identifiers of the plurality of latches with values of a second value or a third value except for the latches to be encoded. For the auxiliary information of each patch included in the current point cloud frame, the auxiliary information can be respectively packaged and encoded into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch with the second value or the third value other than the first patch may be packaged together and encoded into the code stream. And then, packaging the auxiliary information of each patch included in the current point cloud frame one by one and encoding the auxiliary information into a code stream.

In a second possible implementation manner, the first identifier of the first latch in the current point cloud frame and the first identifier of the latch with the second value or the third value other than the first latch may be respectively packaged and encoded into a code stream, and auxiliary information of each latch included in the current point cloud frame may also be respectively packaged and encoded into a code stream. For example, a first identification of a first patch in the current point cloud frame may be encoded into the code stream, and then auxiliary information of the first patch may be encoded into the code stream. And then, encoding auxiliary information of other first-class patches in the current point cloud frame into the code stream one by one. Finally, the remaining patches, i.e., the second class of patch and the third class of patch, may be encoded in a manner of encoding the first identifier first and then encoding the auxiliary information.

In a third possible implementation manner, the first identifiers of the plurality of latches included in the current point cloud frame may be packaged together and encoded into the code stream, where the first identifiers of the plurality of latches include the first identifiers of the latches to be encoded, and the remaining first identifiers of the latches of the plurality of latches that are valued as the second value or the third value except the latches to be encoded. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in the auxiliary information of each patch can be packaged together and encoded into a code stream. Different types of information in the side information for each patch may be individually packetized and encoded into the codestream.

For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch with the second value or the third value other than the first patch may be packaged together and encoded into the code stream. Assuming that the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information, while the auxiliary information of some patches includes two-dimensional information or three-dimensional information, and the auxiliary information of some patches does not include two-dimensional information or three-dimensional information, the rotation transformation information and normal coordinate axis information in the auxiliary information of each patch are packaged together and encoded into a code stream. Two-dimensional information and three-dimensional information in the auxiliary information of each patch are respectively packaged and encoded into a code stream.

The first value, the second value, and the third value may be respectively 10, 11, and 00 in binary, or the first value, the second value, and the third value may be respectively 11, 10, and 00 in binary, etc., where the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame, which is not limited in the embodiments of the present application.

In order to further reduce bit overhead of the code stream, the patches in each point cloud frame in the current point cloud group may be sequentially arranged according to the order of the first class of patches, the second class of patches and the third class of patches, and in this case, if the to-be-encoded patch is the first third class of patches, the to-be-encoded patch may be encoded in a fixed-length encoding manner or in a variable-length encoding manner, which is not limited in the embodiment of the present application. If the patch to be encoded is not the first third type of patch, at this time, the two-dimensional information and the three-dimensional information of the patch to be encoded can be encoded into the code stream by adopting a differential encoding mode according to the three-dimensional information of the first third type of patch, and other information is encoded by adopting the fixed-length encoding mode.

Since different values of the first identifier indicate different types of patch. Thus, different types of patches can be distinguished by the first identifier, and the content included in the auxiliary information encoded into the code stream may be different for different types of patches, and only part of the auxiliary information is encoded into the code stream for the first type of patch and the second type of patch. Thus, the format of the coded stream can be simplified, and the bit overhead of the coded stream can be further reduced, so that the coding efficiency is improved.

In a second aspect, a method for decoding a point cloud is provided, the method comprising: when the point cloud block to be decoded is the first patch in the current point cloud frame, analyzing a first identification of the patch to be decoded from the code stream; when the first mark is a first value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises three-dimensional information of the to-be-decoded patch; reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Because the first identification of the first patch in the current point cloud frame is encoded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, the first identification of the patch to be decoded can be analyzed from the code stream.

In one possible implementation manner, whether the code stream includes the identification information of the patch to be decoded or not may be determined, and if the code stream includes the identification information of the patch to be decoded, the patch to be decoded may be determined to be the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, the patch to be decoded can be determined to be a non-first patch in the current point cloud frame. At this time, it may be determined according to the counter, which patch to decode is the next patch in the current point cloud frame. Wherein the counter starts counting when determining the first patch of the current point cloud frame.

It should be noted that, when the first identifier is the first value, it indicates that the latch to be decoded is a first type latch, that is, there are matched latches in all the point cloud frames in the current point cloud group. At this time, the auxiliary information of the patch to be decoded may be parsed from the code stream, and the auxiliary information includes three-dimensional information of the patch to be decoded, but does not include two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

As an example, the number of the first class of latches can be resolved from the code stream by adopting a method matched with the encoding end. For example, the number of the first class of latches can be resolved from the code stream by using a fixed-length decoding method or a variable-length decoding method.

As can be seen from the method provided in the first aspect, when the first identifier is the first value, the current point cloud frame may be indicated as a non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the patch to be decoded may be decoded by the following two possible implementations.

In a first possible implementation manner, the first identifier of the first latch in the 2 nd point cloud frame in the current point cloud group and the number of the first type latches included in each point cloud frame in the current point cloud group are encoded into the code stream together. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the to-be-decoded latch is the first latch in the current point cloud frame, the first identifier of the to-be-decoded latch may be first parsed from the code stream, and when the first identifier is the first value, the number of the first latches included in each point cloud frame in the current point cloud group is parsed from the code stream. And then analyzing auxiliary information of the patch to be decoded from the code stream.

In a second possible implementation manner, the first identifier of the first type of patch included in the 2 nd point cloud frame in the current point cloud group is encoded into the code stream, at this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, and the patch to be decoded is the first patch, the first identifier of the patch to be decoded can be analyzed from the code stream, and when the first identifier is the first value, the auxiliary information of the patch to be decoded is analyzed from the code stream.

According to the method provided by the first aspect, in the encoding process, the first identification of the non-first type of patch is not encoded into the code stream, so when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, the condition that the patch to be decoded is the non-first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information of the patch to be decoded can be analyzed from the code stream, wherein the auxiliary information comprises the three-dimensional information of the patch to be decoded.

Because the auxiliary information of the code stream is encoded in the coding process of the first type of patch comprises three-dimensional information but does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information, when the to-be-decoded patch is the first type of patch, the auxiliary information analyzed from the code stream comprises the three-dimensional information of the to-be-decoded patch but does not comprise the two-dimensional information, rotation transformation information and normal coordinate axis information of the to-be-decoded patch. At this time, the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded can be obtained according to the auxiliary information of the reference patch of the patch to be decoded.

As an example, decoding may be performed by a method matched with the encoding end, for example, a differential decoding manner may be used to parse the three-dimensional information of the patch to be decoded from the code stream. For other auxiliary information of the patch to be decoded, for example, two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded, identification information of a reference frame of a current point cloud frame can be analyzed from a code stream, so that the reference frame of the current point cloud frame is determined according to the identification information of the reference frame, then the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded are obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. The reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch, for convenience of description, the reference frame of the current point cloud frame is denoted as a first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is denoted as a second reference frame, so that the auxiliary information of the reference patch in the first reference frame may be determined according to the auxiliary information of the corresponding reference patch in the second reference frame. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, and the like. In addition, two-dimensional information, rotation transformation information and normal coordinate axis information of the patch, the position of which is the same as that of the patch to be decoded in the current point cloud frame, can be obtained from the decoded point cloud, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

In other words, when the first identifier is the first value, after the three-dimensional information of the latch to be decoded is parsed from the code stream, the auxiliary information of the latch to be decoded may be further obtained according to the auxiliary information of the reference latch of the latch to be decoded, where the auxiliary information includes two-dimensional information, rotation transformation information and normal coordinate axis information. Thus, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

When the latches in the point cloud are encoded according to the mode that the first type of latches, the second type of latches and the third type of latches are sequentially arranged, the reference latches of the latches to be decoded can be latches with the same positions in the reference frame of the current point cloud frame as the positions of the latches to be decoded in the current point cloud frame.

Based on the above description, if the to-be-decoded latch is the non-first latch in the current point cloud frame, the counter may determine what latch is the to-be-decoded latch in the current point cloud frame. Thus, when the to-be-decoded latch is the nth latch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first latches in the reference point cloud frame of the current point cloud frame, the to-be-decoded latch can be determined to be the non-first latches in the current point cloud frame, so that auxiliary information of the to-be-decoded latch can be analyzed from the code stream, and the auxiliary information comprises three-dimensional information of the to-be-decoded latch.

The number of the first type of latches included in each point cloud frame in the current point cloud group can be determined in two ways. In a first possible implementation manner, after decoding the first identifier of the first patch in the 2 nd point cloud frame in the current point cloud group, the number of the first class patches included in each point cloud frame in the current point cloud group can be directly resolved from the code stream. In a second possible implementation manner, after decoding the first identifier of all the patches of the 2 nd point cloud frame in the current point cloud group, the number of patches with the first identifier as the first value may be determined, so that the number of patches of the first type included in the 2 nd point cloud frame, that is, the number of patches of the first type included in each point cloud frame in the current point cloud group, may be determined. That is, the number of the first class patches included in each point cloud frame in the current point cloud group can be determined through the 2 nd point cloud frame.

In one possible scenario, the 2 nd point cloud frame within the current point cloud group may be the reference point cloud frame of the 3 rd point cloud frame, the 3 rd point cloud frame may be the reference point cloud frame of the 4 th point cloud frame, and so on. In this way, the 3 rd point cloud frame and the point cloud frames after the 3 rd point cloud frame can determine whether n in the nth patch in the current point cloud frame is greater than 1 and less than or equal to the number of the first type patches in the reference point cloud frame of the current point cloud frame according to the number of the first type patches in the reference point cloud frame, so as to determine whether the patch to be decoded is the first type patch.

In addition, because the first identifier of the first patch in the current point cloud frame and the first identifier of the patch with the second value or the third value outside the first patch can be respectively packaged and encoded into the code stream, or can be packaged together and encoded into the code stream. Thus, for the manner of packing and encoding the code stream respectively, if the to-be-decoded latch is the first latch in the current point cloud frame, the first identifier of the to-be-decoded latch can be parsed from the code stream. Of course, for a patch other than the first patch, which takes a second value or a third value, the first identifier of the patch to be decoded may also be parsed from the code stream. That is, the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch with the second value or the third value other than the first patch can be sequentially analyzed. For the manner of packing together and encoding the code stream, if the to-be-decoded latch is the first latch in the current point cloud frame, the first identifiers of the latches included in the current point cloud frame can be parsed from the code stream, wherein the parsed first identifiers include the first identifiers of the to-be-decoded latch. That is, the first identifications of the plurality of latches included in the current point cloud frame can be obtained through one-time analysis without analyzing for a plurality of times. Similarly, when the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packaged together and encoded into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be resolved from the code stream. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one-time analysis, and the analysis is not needed to be performed for multiple times.

As an example, a method matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding mode may be adopted to parse the first identifier of the latch to be decoded from the code stream.

When the to-be-decoded latch is the nth latch in the current point cloud frame, wherein n is greater than 1 and greater than the number of the first latches in the reference point cloud frame of the current point cloud frame, the first identification of the to-be-decoded latch is analyzed from the code stream.

Because the first identifiers are coded into the code stream in the coding process of the second type of patch and the third type of patch, when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the number of the first type of patch in the reference point cloud frame of the current point cloud frame, the first identifier of the patch to be decoded can be analyzed from the code stream.

And when the first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch.

It should be noted that, when the first identifier is the second value, it indicates that the to-be-decoded latch is the second type latch, that is, there is a reference latch matched with the to-be-decoded latch in the reference frame of the current point cloud frame. At this time, auxiliary information of the patch to be decoded may be parsed from the code stream, the auxiliary information including two-dimensional information, three-dimensional information, and identification information of a reference patch of the patch to be decoded, but not including rotation transformation information and normal coordinate axis information.

When the first identifier is the second value, the identification information of the reference patch of the patch to be decoded can be resolved from the code stream by adopting a method matched with the encoding end, for example, a fixed-length decoding mode can be adopted for resolving. For the rotation transformation information and normal coordinate axis information of the to-be-decoded latch, the reference latch of the to-be-decoded latch can be determined through the identification information of the reference latch of the to-be-decoded latch, so that the rotation transformation information and the normal coordinate axis information of the reference latch of the to-be-decoded latch are obtained, and the rotation transformation information and the normal coordinate axis information of the reference latch are used as the rotation transformation information and the normal coordinate axis information of the to-be-decoded latch. For the two-dimensional information and the three-dimensional information of the patch to be decoded, as an example, a method matched with the encoding end can be adopted for decoding, for example, the two-dimensional information and the three-dimensional information of the patch to be decoded can be resolved from the code stream in a differential decoding mode. That is, for the second type of patch, two-dimensional information and three-dimensional information can be analyzed from the code stream, and rotation transformation information and normal coordinate axis information are derived or obtained, so that encoding of the rotation transformation information and the normal coordinate axis information in the code stream is avoided, bit overhead of the code stream is reduced, and meanwhile decoding efficiency is improved.

In other words, when the first identifier is the second value, after the two-dimensional information, the three-dimensional information and the identification information of the reference patch of the patch to be decoded are parsed from the code stream, the rotation transformation information and the normal coordinate axis information of the patch to be decoded can be obtained according to the identification information of the reference patch of the patch to be decoded. Thus, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

And when the first mark is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

It should be noted that, when the first identifier is the third value, it indicates that the to-be-decoded latch is the third type latch, that is, there is no reference latch matched with the to-be-decoded latch in the current point cloud group. At this time, two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded can be resolved from the code stream.

As an example, decoding may be performed by a method matching with the encoding end, for example, two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded may be resolved from the code stream by a fixed-length decoding method, or two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded may be resolved from the code stream by a variable-length decoding method. However, if the auxiliary information of the first third type patch is encoded according to the fixed-length encoding method or the variable-length encoding method, the two-dimensional information and the three-dimensional information of the non-first third type patch are encoded by the differential encoding method, and the other auxiliary information is encoded by the fixed-length encoding method. Then, for the non-first third class of patch, according to the two-dimensional information and the three-dimensional information of the first third class of patch, the two-dimensional information and the three-dimensional information of the patch to be decoded are analyzed from the code stream by adopting a differential decoding mode, and the rotation transformation information and the normal coordinate axis information of the patch to be decoded are analyzed from the code stream by adopting a fixed-length decoding mode. That is, for the third class of patch, two-dimensional information and three-dimensional information of the non-first third class of patch can be analyzed from the code stream, and rotation transformation information and normal coordinate axis information are obtained by deduction or acquisition, so that encoding of the rotation transformation information and the normal coordinate axis information into the code stream is avoided, bit overhead of the code stream is reduced, and meanwhile decoding efficiency is improved.

The first value, the second value, and the third value may be binary values of 10, 11, and 00, respectively, or the first value, the second value, and the third value may be binary values of 11, 10, and 00, respectively, which are not limited in the embodiment of the present application.

Based on the above description, the first class patch, the second class patch and the third class patch in each point cloud frame in the current point cloud group may be sequentially arranged, and at this time, if the patch to be decoded is the first patch in the current point cloud frame, after the first identifier of the patch to be decoded is parsed from the code stream, whether the current point cloud frame is the first frame point cloud in the current point cloud group may be determined according to the value of the first identifier. That is, when the latch to be decoded is the first latch in the current point cloud frame and the first identifier is the third value, the current point cloud frame may be determined to be the first frame point cloud in the current point cloud group. Thus, the length of one point cloud group does not need to be encoded in the code stream.

It should be noted that after the auxiliary information of the latch to be decoded is obtained, that is, after the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the latch to be decoded are obtained, the latch to be decoded can be reconstructed according to, but not limited to, the auxiliary information, and specific information required for reconstructing the latch to be decoded is not limited in the application.

In summary, for the first class of latch and the second class of latch, only part of auxiliary information needs to be analyzed from the code stream, and other auxiliary information can be derived or obtained, so that the format of the analyzed code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.

In a third aspect, a method for encoding a point cloud is provided, the method comprising: acquiring auxiliary information of a point cloud block to be encoded and a first identification of the point cloud block to be encoded; when the first mark is a first value, encoding auxiliary information of the patch to be encoded, the first mark and a second mark of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first mark is a first value, encoding auxiliary information of the patch to be encoded and a second mark of the patch to be encoded into a code stream, wherein the patch to be encoded is a non-first patch in the current point cloud frame; wherein the first identifier is a first value to indicate that the patch to be encoded is a first type of patch; the second mark is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; or the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded comprises the three-dimensional information.

The first class of patches have patches with matching relations in all point cloud frames in the current point cloud group, two-dimensional information, rotation transformation information and normal coordinate axis information among the first class of patches in the current point cloud group are the same, and three-dimensional information may not be the same, but in some cases, the three-dimensional information of the first class of patches may also be the same. In this case, when the first flag is a first value, the patch to be encoded may be indicated as a first type of patch, and the information encoded into the code stream may further include a second flag.

It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded is already encoded into the code stream. The syntax element of the second flag may be override_3d_shift_data_flag. When the second identifier is a fourth value, the three-dimensional information of the patch to be encoded may be indicated to be not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so that the three-dimensional information of the patch to be encoded does not need to be encoded into the code stream again, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. When the second flag is a fifth value, it may indicate that the three-dimensional information of the patch to be encoded is already encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded, so that the three-dimensional information of the patch to be encoded needs to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include the three-dimensional information. The specific encoding method is not limited herein, and for example, a differential encoding method or a fixed-length encoding method may be used. The syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.

When the first identifier is the second value, the to-be-encoded latch may be indicated as a second type latch. That is, there is a reference patch matching the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of a reference patch of the patch to be encoded.

When the first identifier is a third value, the to-be-encoded patch may be indicated as a third type patch. That is, there is no reference patch matching the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Because the number of the first type of patches in the current point cloud group in each point cloud frame can be the same, when the first mark is a first value and the patch to be encoded is the first patch in the current point cloud frame, the condition that the patch to be encoded is the first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information, the first mark and the second mark of the patch to be encoded can be encoded into a code stream; when the first mark is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the to-be-encoded patch is indicated to be a non-first type patch in the current point cloud frame, auxiliary information and the second mark of the to-be-encoded patch can be encoded into the code stream, and the first mark of the to-be-encoded patch is not encoded into the code stream, so that the format of the encoded code stream can be simplified. The second identifier is an identifier for indicating whether the three-dimensional information of the patch to be encoded is encoded into the code stream. When the second identifier is a fourth value, the three-dimensional information indicating the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so that the three-dimensional information of the patch to be encoded does not need to be encoded into the code stream again, that is, the auxiliary information of the patch to be encoded does not include the three-dimensional information, so that the bit overhead of the code stream can be further reduced, and the encoding efficiency is improved.

In a fourth aspect, a method for decoding a point cloud is provided, the method including: when the point cloud block to be decoded is the first patch in the current point cloud frame, analyzing a first identification of the patch to be decoded from the code stream; when the first identifier is a first value, a second identifier of the patch to be decoded is analyzed from the code stream; when the second mark is a fourth value, determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is more than 1 and less than or equal to the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing a second identification of the to-be-decoded patch from the code stream; when the second mark is a fourth value, determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that, when the second flag is the fourth value, it may indicate that the three-dimensional information of the latch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the to-be-decoded latch is the same as the three-dimensional information of the reference latch of the to-be-decoded latch, and at this time, the auxiliary information of the to-be-decoded latch may be determined according to the auxiliary information of the reference latch of the to-be-decoded latch. Specifically, the auxiliary information of the reference patch of the patch to be decoded may be used as the auxiliary information of the patch to be decoded. When the second identifier is the fifth value, the three-dimensional information of the patch to be decoded can be indicated to be encoded into the code stream, and the auxiliary information of the patch to be decoded comprises the three-dimensional information. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes three-dimensional information of the patch to be decoded. As an example, the auxiliary information of the patch to be decoded may be parsed from the code stream by differential decoding. Reference is made to the foregoing for specific details.

Since the second identifiers of the non-first type of latches in the current point cloud frame are all encoded into the code stream, when the latch to be decoded is the nth latch in the current point cloud frame, n is greater than 1 and greater than the number of the first type of latches in the reference point cloud frame of the current point cloud frame, that is, when the latch to be decoded is the non-first type of latch, the second identifier of the latch to be decoded can be analyzed from the code stream.

Because the first identification of the first patch in the current point cloud frame is encoded into the code stream, when the point cloud patch to be decoded is the first patch in the current point cloud frame, the first identification of the patch to be decoded is analyzed from the code stream, and when the first identification is a first value, the first identification of the patch to be decoded is indicated to be the first type of patch, and at the moment, the second identification of the patch to be decoded can be analyzed from the code stream. When the to-be-decoded latch is the nth latch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first latches in the reference point cloud frame of the current point cloud frame, indicating that the to-be-decoded latch is not the first latch, and analyzing the second identification of the to-be-decoded latch from the code stream. And when the second mark is a fourth value, indicating that the three-dimensional information of the patch to be decoded is not coded into the code stream. That is, the three-dimensional information of the to-be-decoded latch is the same as the three-dimensional information of the reference latch of the to-be-decoded latch, and at this time, the three-dimensional information of the reference latch of the to-be-decoded latch can be used as the three-dimensional information of the to-be-decoded latch, so that the three-dimensional information of the to-be-decoded latch does not need to be parsed from the code stream, the format of the parsed code stream can be simplified, and the bit overhead of the code stream can be reduced, thereby improving the decoding efficiency.

In a fifth aspect, a point cloud encoding method is provided, the method including: acquiring auxiliary information of a point cloud block to be encoded and a first identification of the point cloud block to be encoded; when the first identifier is a first value, encoding the first identifier of the patch to be encoded into the code stream; when the first identifier is a second value or a third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into a code stream; wherein the first identifier is a first value to indicate that the patch to be encoded is a first type of patch; the first mark is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be encoded; the first mark is a third value to indicate that the patch to be encoded is a third type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

When the first identifier is a first value, the patch to be encoded is indicated as a first type of patch. In one possible implementation manner, the first type of patch has patches with matching relation in all point cloud frames in the current point cloud group, and two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information among the first type of patches in the current point cloud group are the same. Therefore, in order to reduce the bit overhead of the code stream and further improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is coded into the code stream, and auxiliary information of the patch to be coded is not coded into the code stream.

Because the two-dimensional information, the rotation transformation information, the normal coordinate axis information and the three-dimensional information among the first type of the patch in the current point cloud group can be the same, when the first mark of the patch to be encoded is a first value, the patch to be encoded is indicated to be the first type of the patch, at the moment, the first mark of the patch to be encoded can be encoded into a code stream, and auxiliary information of the patch to be encoded does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified. In addition, the value of the first identifier can be a first value, a second value and a third value. Different values indicate different types of patches. Thus, different types of patches can be distinguished by the first identifier, and the content included in the auxiliary information encoded into the code stream may be different for different types of patches, and only part of the auxiliary information is encoded into the code stream for the first type of patch and the second type of patch. Thus, the format of the coded stream can be simplified, and the bit overhead of the coded stream can be further reduced, so that the coding efficiency is improved.

In a sixth aspect, a method for decoding a point cloud is provided, the method including: analyzing a first identification of a point cloud block patch to be decoded from a code stream; when the first mark is a first value, determining auxiliary information of the patch to be decoded according to auxiliary information of a reference patch of the patch to be decoded, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; when the first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch; when the first identifier is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Since the first identifications of all the patches in the current point cloud frame can be respectively packed and encoded into the code stream, the first identifications can also be packed together and encoded into the code stream. Thus, for the manner of packing and encoding the code stream respectively, if the to-be-decoded latch is the first latch in the current point cloud frame, the first identifier of the to-be-decoded latch can be parsed from the code stream. Of course, for non-first patches, the first identification of the patch to be decoded may also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame may be parsed in turn. For the manner of packing together and encoding the code stream, if the to-be-decoded latch is the first latch in the current point cloud frame, the first identifiers of the latches included in the current point cloud frame can be parsed from the code stream, wherein the parsed first identifiers include the first identifiers of the to-be-decoded latch. That is, the first identifications of the plurality of latches included in the current point cloud frame can be obtained through one-time analysis without analyzing for a plurality of times.

Because the first type of patch does not encode auxiliary information into the code stream in the encoding process, when the first identifier is a first value, that is, when the patch to be decoded is the first type of patch, two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded can be obtained according to the auxiliary information of the reference patch of the patch to be decoded.

As an example, the identification information of the reference frame of the current point cloud frame may be first parsed from the code stream, so that the reference frame of the current point cloud frame is determined according to the identification information of the reference frame, then the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the reference patch of the patch to be decoded are obtained from the reference frame of the current point cloud frame, and the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the reference patch are used as the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded. The reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch, for convenience of description, the reference frame of the current point cloud frame is denoted as a first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is denoted as a second reference frame, so that the auxiliary information of the reference patch in the first reference frame may be determined according to the auxiliary information of the corresponding reference patch in the second reference frame. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, and the like. In addition, for the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded, the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch, which are positioned in the same position as the patch to be decoded in the current point cloud frame, can be obtained from the decoded point cloud, and the obtained two-dimensional information, the three-dimensional information, the rotation transformation information and the obtained normal coordinate axis information are used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.

When the patches in each point cloud frame in the current point cloud group are coded according to the sequence arrangement of the first class of patches, the second class of patches and the third class of patches, the reference patch of the patch to be decoded can be the patch with the same position in the reference frame of the current point cloud frame as the position of the patch to be decoded in the current point cloud frame.

The auxiliary information of the second type of patch is encoded into the code stream in the encoding process and comprises two-dimensional information, three-dimensional information and identification information of the reference patch, but does not comprise rotation transformation information and normal coordinate axis information, so when the patch to be decoded is the second type of patch, the auxiliary information analyzed from the code stream comprises the two-dimensional information, the three-dimensional information and the identification information of the reference patch of the patch to be decoded, but does not comprise the rotation transformation information and the normal coordinate axis information of the patch to be decoded. At this time, rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.

In summary, for the first class of patch, only the first identifier needs to be resolved from the code stream, other auxiliary information may be derived or obtained, and for the second class of patch, only part of the auxiliary information needs to be resolved from the code stream, other auxiliary information may also be derived or obtained. Thus, the format of the analysis code stream can be simplified, and the bit cost of the code stream can be reduced, thereby improving the decoding efficiency.

In a seventh aspect, there is provided a point cloud encoding apparatus, the apparatus comprising: the point cloud block patch information acquisition module is used for acquiring auxiliary information of the point cloud block patch to be encoded and a first identification of the patch to be encoded; the auxiliary information coding module is used for coding the auxiliary information of the patch to be coded and the first identifier into a code stream when the first identifier is a first value, and the patch to be coded is the first patch in the current point cloud frame; or an auxiliary information encoding module, configured to encode auxiliary information of a patch to be encoded into a code stream when the first identifier is a first value, where the patch to be encoded is a non-first patch in the current point cloud frame; the first mark is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded comprises three-dimensional information.

The current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Wherein, this auxiliary information coding module is still used for:

when the first identifier is a second value or a third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into a code stream;

the first mark is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be encoded;

The first mark is a third value to indicate that the patch to be encoded is a third type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Wherein, this auxiliary information coding module is still used for:

encoding first identifiers of a plurality of patches included in the current point cloud frame into a code stream, wherein the first identifiers of the plurality of patches comprise first identifiers of the patches to be encoded, and the first identifiers of the plurality of patches which are left in the first identifiers of the plurality of patches except the first identifiers of the patches to be encoded are second values or third values.

Wherein the first value, the second value and the third value are binary 10, 11 and 00 respectively,

alternatively, the first, second and third values are binary 11, 10 and 00, respectively.

The first mark of the patch to be encoded is a third value, and the patch to be encoded is the first patch in the current point cloud frame.

The reference patch to be coded is included in a reference frame of the current point cloud frame.

The first type of the patch is a global matching patch in the current group, the second type of the patch is a local matching patch in the current group, and the third type of the patch is a non-matching patch in the current group.

An eighth aspect provides a point cloud decoding apparatus, the apparatus comprising: the auxiliary information decoding module is used for analyzing a first identification of the to-be-decoded patch from the code stream when the to-be-decoded point cloud block patch is the first patch in the current point cloud frame; the auxiliary information decoding module is further used for analyzing auxiliary information of the patch to be decoded from the code stream when the first identifier is a first value, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; the auxiliary information decoding module is further configured to parse auxiliary information of the to-be-decoded latch from the code stream when the to-be-decoded latch is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first-class latches in the reference point cloud frame of the current point cloud frame, and the auxiliary information includes three-dimensional information of the to-be-decoded latch; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein, this auxiliary information decoding module is also used for:

when the to-be-decoded patch is the nth patch in the current point cloud frame, wherein n is greater than 1 and greater than the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing a first identification of the to-be-decoded patch from the code stream;

when the first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch;

Wherein, this auxiliary information decoding module is also used for:

when the to-be-decoded patch is the first patch in the current point cloud frame, analyzing first identifiers of a plurality of patches included in the current point cloud frame from the code stream, wherein the analyzed first identifiers comprise first identifiers of the to-be-decoded patch, and the first identifiers of the plurality of first identifiers except the first identifiers of the to-be-decoded patch are second values or third values.

When the to-be-decoded latch is the first latch in the current point cloud frame and the first identifier is the third value, the current point cloud frame is the first point cloud frame.

Wherein when the first flag is a first value, the side information decoding module is further configured to:

acquiring two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded according to auxiliary information of a reference patch of the patch to be decoded;

the reconstruction module is also for:

reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

Wherein when the first flag is a second value, the side information decoding module is further configured to:

acquiring rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;

the reconstruction module is also for:

The reference patch to be decoded is included in a reference frame of the current point cloud frame.

In a ninth aspect, there is provided a point cloud encoding apparatus, the apparatus comprising: the point cloud block patch information acquisition module is used for acquiring auxiliary information of the point cloud block patch to be encoded and a first identification of the patch to be encoded; the auxiliary information coding module is used for coding auxiliary information of the patch to be coded, the first identifier and the second identifier of the patch to be coded into a code stream when the first identifier is a first value, and the patch to be coded is the first patch in the current point cloud frame; or an auxiliary information coding module, configured to code auxiliary information of a patch to be coded and a second identifier of the patch to be coded into a code stream when the first identifier is a first value, where the patch to be coded is a non-first patch in the current point cloud frame; wherein the first identifier is a first value to indicate that the patch to be encoded is a first type of patch; the second mark is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; or the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded comprises the three-dimensional information.

In a tenth aspect, there is provided a point cloud decoding apparatus, the apparatus comprising: the auxiliary information decoding module is used for analyzing a first identification of the to-be-decoded patch from the code stream when the to-be-decoded point cloud block patch is the first patch in the current point cloud frame; the auxiliary information decoding module is further used for analyzing a second identifier of the patch to be decoded from the code stream when the first identifier is a first value; the auxiliary information decoding module is further used for determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is a fourth value; when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; the auxiliary information decoding module is further configured to parse a second identifier of the latch to be decoded from the code stream when the latch to be decoded is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first latches in the reference point cloud frame of the current point cloud frame; the auxiliary information decoding module is further used for determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is a fourth value; when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

In an eleventh aspect, there is provided a point cloud encoding apparatus, the apparatus comprising: the point cloud block patch information acquisition module is used for acquiring auxiliary information of the point cloud block patch to be encoded and a first identification of the patch to be encoded; the auxiliary information coding module is used for coding the first identification of the patch to be coded into the code stream when the first identification is a first value; the auxiliary information coding module is further used for coding auxiliary information of the patch to be coded and the first identifier into a code stream when the first identifier is a second value or a third value; wherein the first identifier is a first value to indicate that the patch to be encoded is a first type of patch; the first mark is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be encoded; the first mark is a third value to indicate that the patch to be encoded is a third type of patch, and the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

In a twelfth aspect, there is provided a point cloud decoding apparatus, the apparatus comprising: the auxiliary information decoding module is used for analyzing a first identification of the point cloud block patch to be decoded from the code stream; the auxiliary information decoding module is further used for determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the first identifier is a first value, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; the auxiliary information decoding module is further used for analyzing auxiliary information of the patch to be decoded from the code stream when the first identifier is a second value, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be decoded; the auxiliary information decoding module is further used for analyzing auxiliary information of the patch to be decoded from the code stream when the first identifier is a third value, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; and the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

the reconstruction module is also for:

In a thirteenth aspect, an apparatus for encoding point cloud data is provided, the apparatus may include: memory and encoder. The memory is used for storing point cloud data. The encoder is configured to perform the point cloud encoding method provided by the above-described first aspect or any of the possible designs of the first aspect, or the encoder is configured to perform the point cloud encoding method provided by the above-described third aspect or any of the possible designs of the third aspect, or the encoder is configured to perform the point cloud encoding method provided by the above-described fifth aspect or any of the possible designs of the fifth aspect.

In a fourteenth aspect, an apparatus for decoding point cloud data is provided, the apparatus may include: a memory and a decoder. The memory is used for storing point cloud data in the form of a code stream. The decoder is configured to perform the point cloud decoding method provided by the second aspect or any one of the possible designs of the second aspect, or the decoder is configured to perform the point cloud decoding method provided by the fourth aspect or any one of the possible designs of the fourth aspect, or the decoder is configured to perform the point cloud decoding method provided by the sixth aspect or any one of the possible designs of the sixth aspect.

A fifteenth aspect provides an encoding apparatus, comprising: a memory and a processor; wherein the memory is for storing program code; the processor is configured to invoke the program code to perform the point cloud encoding method provided in the first aspect, the third aspect, or the fifth aspect.

A sixteenth aspect provides a decoding apparatus, comprising: a memory and a processor; wherein the memory is for storing program code; the processor is configured to invoke the program code to perform the point cloud decoding method provided in the second aspect, the fourth aspect, or the sixth aspect.

The present application also provides a computer readable storage medium comprising program code which when run on a computer causes the computer to perform any one of the point cloud encoding methods provided as described above for the first aspect and its possible designs, or for the third aspect and its possible designs, or for the fifth aspect and its possible designs.

The present application also provides a computer readable storage medium comprising program code which when run on a computer causes the computer to perform any one of the point cloud decoding methods provided in the second aspect and its possible designs, or the fourth aspect and its possible designs, or the sixth aspect and its possible designs.

The present application also provides a computer program product which, when run on a computer, causes the computer to perform the first aspect or any one of the possible designs of the first aspect, or to perform the third aspect or any one of the possible designs of the third aspect, or to perform some or all of the steps of the point cloud encoding method provided by the fifth aspect or any one of the possible designs of the fifth aspect.

The present application also provides a computer program product which when run on a computer causes the computer to perform the second aspect or any one of the possible designs of the second aspect, or to perform the fourth aspect or any one of the possible designs of the fourth aspect, or to perform some or all of the steps of the point cloud decoding method provided by the sixth aspect or any one of the possible designs of the sixth aspect.

It should be appreciated that the advantages of any of the above-provided codec device, computer-readable storage medium and computer program product may correspond to the advantages of the method embodiments provided with reference to the corresponding aspects above, and are not described in detail.

The technical scheme that this application provided can bring following beneficial effect at least:

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are acquired first. Because the number of the first type of patches in the current point cloud group in each point cloud frame can be the same, when the first mark is a first value and the patch to be encoded is the first patch in the current point cloud frame, the condition that the patch to be encoded is the first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information and the first mark of the patch to be encoded can be encoded into a code stream; when the first mark is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the to-be-encoded patch is indicated to be a non-first type patch in the current point cloud frame, and at the moment, auxiliary information of the to-be-encoded patch can be encoded into a code stream without encoding the first mark of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified. The bit overhead of the code stream is further reduced, thereby improving the coding efficiency.

Drawings

Fig. 1 is a schematic block diagram of a point cloud decoding system according to an embodiment of the present application.

Fig. 2 is a schematic block diagram of an encoder that may be used in embodiments of the present application.

Fig. 3 is a schematic diagram of a point cloud that may be suitable for use in embodiments of the present application.

FIG. 4 is a schematic diagram of a patch of a point cloud that may be suitable for use in embodiments of the present application.

Fig. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application.

Fig. 6 is a schematic block diagram of a decoder that may be used in embodiments of the present application.

Fig. 7 is a schematic diagram of a frame set that may be used in embodiments of the present application.

Fig. 8 is a schematic diagram of a first current point cloud set according to an embodiment of the present application.

Fig. 9 is a flowchart of a first point cloud encoding method according to an embodiment of the present application.

Fig. 10 is a schematic diagram of a second current point cloud set according to an embodiment of the present application.

Fig. 11 is a flowchart of a first point cloud decoding method according to an embodiment of the present application.

Fig. 12 is a flowchart of a second point cloud encoding method according to an embodiment of the present application.

Fig. 13 is a flowchart of a second point cloud decoding method according to an embodiment of the present application.

Fig. 14 is a flowchart of a third point cloud encoding method according to an embodiment of the present application.

Fig. 15 is a flowchart of a third method for decoding point cloud according to an embodiment of the present application.

Fig. 16 is a schematic block diagram of a first encoding apparatus according to an embodiment of the present application.

Fig. 17 is a schematic block diagram of a first decoding apparatus provided in an embodiment of the present application.

Fig. 18 is a schematic block diagram of a second encoding apparatus according to an embodiment of the present application.

Fig. 19 is a schematic block diagram of a second decoding apparatus according to an embodiment of the present application.

Fig. 20 is a schematic block diagram of a third encoding apparatus provided in an embodiment of the present application.

Fig. 21 is a schematic block diagram of a third decoding apparatus provided in an embodiment of the present application.

Fig. 22 is a schematic block diagram of one implementation of a decoding apparatus for embodiments of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The term "plurality" in the embodiments of the present application refers to two or more. In the description of the present application, "/" means or, unless otherwise indicated, for example, a/B may represent a or B; "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", and the like are used to distinguish the same item or similar items having substantially the same function and effect. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.

Before explaining the embodiments of the present application in detail, an implementation environment of the embodiments of the present application is described:

an implementation environment provided by the embodiment of the application includes a point cloud decoding system. Fig. 1 is a schematic block diagram of a point cloud decoding system according to an embodiment of the present application. Wherein the term "point cloud coding" or "coding" may refer generally to point cloud encoding or point cloud decoding. Referring to fig. 1, the point cloud decoding system includes a source device 10, a destination device 20, a link 30, and a storage device 40. Wherein the source device 10 may generate encoded point cloud data. Thus, the source device 10 may also be referred to as a point cloud encoding device. The destination device 20 may decode the encoded point cloud data generated by the source device 10. Thus, the destination device 20 may also be referred to as a point cloud decoding device. The link 30 may receive encoded point cloud data generated by the source device 10 and may transmit the encoded point cloud data to the destination device 20. The storage device 40 may receive the encoded point cloud data generated by the source device 10 and may store the encoded point cloud data such that the destination device 20 may obtain the encoded point cloud data directly from the storage device 40. Alternatively, the storage device 40 may correspond to a file server or another intermediate storage device that may store encoded point cloud data generated by the source device 10, such that the destination device 20 may store the encoded point cloud data via streaming or download the storage device 40.

The source device 10 and the destination device 20 may each include one or more processors and memory coupled to the one or more processors, which may include random access memory (random access memory, RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, and the like. For example, source device 10 and destination device 20 may each comprise a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an on-board computer, or the like.

The link 30 may include one or more media or devices capable of transmitting encoded point cloud data from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to transmit encoded point cloud data directly to the destination device 20 in real-time. In embodiments of the present application, the source device 10 may modulate the encoded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may transmit the modulated point cloud data to the destination device 20. The one or more communication media may include wireless and/or wired communication media, for example, the one or more communication media may include a Radio Frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, which may be a local area network, a wide area network, or a global network (e.g., the internet), etc. The one or more communication media may include a router, switch, base station, or other device that facilitates communication from source device 10 to destination device 20, etc., as embodiments of the present application are not specifically limited in this regard.

In one possible implementation, the storage device 40 may store the received encoded point cloud data transmitted by the source device 10, and the destination device 20 may obtain the encoded point cloud data directly from the storage device 40. As such, storage device 40 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, blu-ray disc, digital versatile disc (digital versatile disc, DVD), read only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded point cloud data, or the like.

In one possible implementation, the storage device 40 may correspond to a file server or another intermediate storage device that may hold encoded point cloud data generated by the source device 10, and the destination device 20 may store the point cloud data via streaming or downloading the storage device 40. The file server may be any type of server capable of storing and transmitting encoded point cloud data to the destination device 20. In one possible implementation, the file server may comprise a web server, a file transfer protocol (file transfer protocol, FTP) server, a network attached storage (network attached storage, NAS) device, or a local disk drive, among others. The destination device 20 may obtain the encoded point cloud data through any standard data connection, including an internet connection. Any standard data connection may include a wireless channel (e.g., wi-Fi connection), a wired connection (e.g., digital subscriber line (digital subscriber line, DSL), cable modem, etc.), or a combination of both suitable for obtaining encoded point cloud data stored on a file server. The transmission of encoded point cloud data from storage 40 may be a streaming transmission, a download transmission, or a combination of both.

The point cloud decoding system shown in fig. 1 is only one possible implementation manner, and the technology of the present application may be applied not only to the source device 10 that may encode a point cloud and the destination device 20 that may decode encoded point cloud data shown in fig. 1, but also to other devices that may encode a point cloud and decode encoded point cloud data, which is not limited in particular in the embodiment of the present application.

In the point cloud decoding system shown in fig. 1, the source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, which may also be referred to as a transmitter. The data sources 120 may include point cloud capture devices (e.g., cameras, etc.), point cloud archives containing previously captured point cloud data, point cloud feed interfaces for receiving point cloud data from point cloud content providers, and/or computer graphics systems for generating point cloud data, or a combination of these sources of point cloud data.

The data source may send a point cloud to encoder 100, and encoder 100 may encode the received point cloud sent by data source 120 to obtain encoded point cloud data. The encoder may send the encoded point cloud data to an output interface. In some embodiments, the source device 10 sends the encoded point cloud data directly to the destination device 20 via the output interface 140. In other embodiments, the encoded point cloud data may also be stored onto storage device 40 for later retrieval by destination device 20 and use in decoding and/or playback.

In the embodiment of fig. 1, destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, input interface 240 includes a receiver and/or a modem. The input interface 240 may receive the encoded point cloud data via the link 30 and/or from the storage device 40 and then send it to the decoder 200, and the decoder 200 may decode the received encoded point cloud data to obtain decoded point cloud data. The decoder may transmit the decoded point cloud data to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. In general, the display device 220 displays the decoded point cloud data. The display device 220 may be any one of various types of display devices, for example, the display device 220 may be a liquid crystal display (liquid crystal display, LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

Although not shown in fig. 1, in some aspects, encoder 100 and decoder 200 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX units may be compliant with ITU h.223 multiplexer protocol, or other protocols such as user datagram protocol (user datagram protocol, UDP), if applicable.

Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (digital signal processing, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field-programmable gate arrays (field-programmable gate array, FPGA), discrete logic, hardware, or any combination thereof. If the present application is implemented in part in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium and the instructions may be executed in hardware using one or more processors to implement the techniques of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (codec) in the respective device.

The present application may generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and/or other data used to decode the compressed point cloud data. This transfer may occur in real time or near real time. Alternatively, this communication may occur over a period of time, such as may occur when syntax elements are stored to a computer-readable storage medium in an encoded bitstream when encoded, which the decoding device may then retrieve at any time after the syntax elements are stored to such medium.

Fig. 2 is a schematic block diagram of an encoder 100 provided in an embodiment of the present application. Fig. 2 illustrates an example MPEG (Moving Pictrue Expert Group) point cloud compression (Point Cloud Compression, PCC) encoding framework. Referring to fig. 2, the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, an image or video-based encoding module 107, an occupancy map encoding module 108, an auxiliary information encoding module 109, a multiplexing module 110, a point cloud occupancy map downsampling module 111, a point cloud occupancy map filling module 112, a point cloud reconstruction module 113, and a point cloud filtering module 114.

The point cloud block information generating module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds, and for convenience of description, the following description will take a current point cloud frame in the current point cloud group as an example. The point cloud block information generation module 101 may determine a three-dimensional coordinate of each point included in the current point cloud frame in a three-dimensional space coordinate system, and a normal direction vector of each point in the three-dimensional space, and divide the current point cloud frame into a plurality of patches (point cloud blocks) according to the determined normal direction vector of each point in the three-dimensional space and a predefined projection plane. Each patch includes one or more points in the current point cloud frame. The point cloud block information generation module 101 may also determine an occupancy map for each patch and a depth map for each patch. The point cloud block information generating module 101 may further determine three-dimensional information of each patch and normal coordinate axis information of each patch, which may be auxiliary information of each patch. In addition, the point cloud block information generation module 101 may divide the patch into three types, that is, a first type of patch, a second type of patch, and a third type of patch, respectively. And indicates the type of each patch by the first identification. The point cloud block information generating module 101 may send the first identifier of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 to perform encoding, which may also be referred to as performing compression encoding. The three-dimensional information of each patch may include an offset of the patch to be encoded in the 3D space along a tangential direction, an offset of the patch to be encoded in the 3D space along a double tangential direction, and an offset of the patch to be encoded in the 3D space along a normal direction. The point cloud block information generation module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102. In addition, the point cloud block information generating module 101 may also send the depth map of each patch to the depth map generating module 103.

The packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101, to obtain an occupancy map of the current point cloud frame. Specifically, the packaging module 102 may arrange the occupancy maps of each patch in a specific order, for example, in descending (or ascending) order of the occupancy maps of each patch, and then sequentially insert the occupancy maps of each patch into the available area of the occupancy map of the current point cloud frame according to the arranged occupancy maps of each patch, to obtain the occupancy map of the current point cloud frame and packaging information of the patches. The packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding. The package information of each patch may include two-dimensional information of each patch in an occupancy map of the current point cloud frame, identification information of a reference patch of each patch, rotation transformation information of each patch, and identification information of a reference frame of the current point cloud frame. The two-dimensional information of each patch in the occupancy map of the current point cloud frame may be simply referred to as two-dimensional information of each patch. The two-dimensional information of each patch may include an offset of each patch in a u-axis direction in the current point cloud frame occupancy map, an offset of each patch in a v-axis direction in the current point cloud frame occupancy map, a width of each patch in the current point cloud frame occupancy map, and a height of each patch in the current point cloud frame occupancy map. In addition, the packaging module 102 may also send the package information of the patch to the depth map generating module 103 and the point cloud occupancy map downsampling module 111.

It should be noted that, in order to more intuitively understand the point cloud, the latches of the point cloud, and the occupancy map of the point cloud related to the technology of the present application, refer to fig. 3, fig. 4, and fig. 5, fig. 3 is a schematic diagram of a frame of point cloud applicable to an embodiment of the present application, fig. 4 is a schematic diagram of the latches of the frame of point cloud, and fig. 5 is a schematic diagram of the occupancy map of each latch of the frame of point cloud shown in fig. 4, where the occupancy map of the frame of point cloud is obtained by packing by the packing module 102. The point cloud shown in fig. 3 may be a current point cloud frame in the embodiment of the present application, the patch of the point cloud shown in fig. 4 may be a patch of the current point cloud frame in the embodiment of the present application, and the occupancy map of the point cloud shown in fig. 5 may be an occupancy map of the current point cloud frame in the embodiment of the present application.

The auxiliary information encoding module 109 may encode the current point cloud frame and each patch included in the current point cloud frame after receiving the packing information of the occupancy map and the patches of the current point cloud frame sent by the packing module 102 and the first identifier of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101, obtain a code stream including the encoded auxiliary information, and then send the obtained code stream including the encoded auxiliary information to the multiplexing module 110. Wherein, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the package information of the patch may also be referred to as auxiliary information of each patch.

The depth map generation module 103 may generate a depth map of the current point cloud frame according to the packing information of each patch and the depth map of each patch after receiving the packing information of the patch transmitted by the packing module 102 and the depth map packing of each patch transmitted by the point cloud block information generation module 101. And then sending the generated depth map of the current point cloud frame to a depth map filling module 105 so as to fill blank pixel points in the depth map of the current point cloud frame, thereby obtaining the depth map of the filled current point cloud frame. The depth map filling module 105 may send the obtained depth map of the filled current point cloud frame to the image or video based encoding module 107 to encode the depth map of the filled current point cloud frame based on the image or video to obtain a reconstructed depth map of the current point cloud and a code stream including the depth map of the encoded current point cloud frame, and may send the obtained depth map of the reconstructed current point cloud frame to the point cloud reconstruction module 113 and the code stream including the depth map of the encoded current point cloud frame to the multiplexing module 110.

The point cloud occupancy map downsampling module 111 may downsample the occupancy map of the current point cloud frame received from the packaging module 102 to obtain an occupancy map of the current point cloud frame with low resolution. The point cloud occupancy map downsampling module 111 may then send the occupancy map of the current point cloud frame with the low resolution to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112. The occupancy map encoding module 108 may encode the occupancy map of the received low-resolution current point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 may further send the code stream including the occupancy map of the encoded low-resolution current point cloud frame to the multiplexing module 110. The point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the current point cloud frame with the low resolution, and obtains the occupancy map of the filled current point cloud frame with the original resolution.

The point cloud reconstruction module 113 may reconstruct the geometry of the current point cloud frame according to the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, the depth map of the reconstructed current point cloud frame sent by the image or video-based encoding module 107, the packing information of the patch, and the auxiliary information of the patch, so as to output a reconstructed point cloud, and in addition, the point cloud reconstruction module 113 may also output the correspondence between the reconstructed point in the reconstructed point cloud and the patch, and the packing position of the reconstructed point in the reconstructed point cloud. The point cloud reconstruction module 113 may send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 may also send the packaging position of the reconstructed point in the reconstructed point cloud to the texture generating module 104.

The point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113, and the correspondence between the reconstructed points and the patch in the reconstructed point cloud. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be called a smooth reconstructed point cloud. Alternatively, the point cloud filter block 114 may smooth the reconstructed point cloud.

After receiving the smoothed reconstructed point cloud sent by the point cloud filtering module 114, the packing positions of the reconstructed points in the reconstructed point cloud sent by the point cloud reconstructing module 113, and the current point cloud frame sent by the data source 120, the texture map generating module 104 may generate a texture map of the current point cloud frame according to the smoothed reconstructed point cloud, the packing positions of the reconstructed points in the reconstructed point cloud, and the current point cloud frame, and may send the generated texture map of the current point cloud frame to the texture map filling module 106 to fill blank pixel points in the texture map of the current point cloud frame, so as to obtain a texture map of the filled current point cloud frame. The texture map filling module 106 may send the texture map of the obtained filled current point cloud frame to the image or video based encoding module 107 to perform image or video based encoding on the texture map of the filled current point cloud frame, to obtain a code stream including the texture map of the reconstructed current point cloud frame. The image or video based encoding module 107 may also send the resulting code stream including the texture map of the reconstructed current point cloud frame to the multiplexing module 110.

Through the above description, the image or video based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 may send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 may combine the received code streams into a combined code stream and send the combined code stream to the output interface 140. The output interface 140 may send the combined code stream to the decoder 200.

It should be understood that the encoder 100 shown in fig. 2 is merely one embodiment provided herein, and that in a particular implementation, the encoder 100 may include more or fewer modules than those shown in fig. 2. The embodiment of the present application is not particularly limited thereto.

Fig. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the present application. Fig. 6 illustrates an MPEG PCC decoding framework. Referring to fig. 6, the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map coding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, a point cloud geometry information reconstruction module 206, a point cloud filtering module 207, and a point cloud texture information reconstruction module 208.

The demultiplexing module 201 may receive the combined code stream transmitted by the output interface 140 of the encoder 100 through the input interface 204 and transmit the combined code stream to a corresponding decoding module. Specifically, the demultiplexing module 201 sends a code stream including a texture map of the encoded current point cloud frame and a code stream including a depth map of the encoded current point cloud frame to the image or video-based decoding module 202, sends a code stream including an occupancy map of the encoded low resolution current point cloud frame to the occupancy map coding module 203, and sends a code stream including encoded auxiliary information to the auxiliary information decoding module 204.

The image or video based decoding module 202 may decode the received code stream including the texture map of the encoded current point cloud frame and the code stream including the depth map of the encoded current point cloud frame to obtain the texture map information of the reconstructed current point cloud frame and the depth map information of the reconstructed current point cloud frame, and may send the texture map information of the reconstructed current point cloud frame to the texture information reconstruction module 208 of the point cloud and the depth map information of the reconstructed current point cloud frame to the geometric information reconstruction module 206 of the point cloud. The occupancy map encoding module 203 may decode a received code stream including the occupancy map of the encoded low-resolution current point cloud frame to obtain occupancy map information of the reconstructed low-resolution current point cloud frame, and send the occupancy map information of the reconstructed low-resolution current point cloud frame to the point cloud occupancy map filling module 205. The point cloud occupancy map filling module 205 may obtain the occupancy map information of the reconstructed current point cloud frame with the original resolution according to the occupancy map information of the reconstructed current point cloud frame with the low resolution, and then send the occupancy map information of the reconstructed current point cloud frame with the original resolution to the geometric information reconstruction module 206 of the point cloud. For convenience of description, the subsequent unification is to reconstruct the occupancy map information of the current point cloud frame with the original resolution, which is abbreviated as the occupancy map information of the reconstructed current point cloud frame.

The side information decoding module 204 may decode the received code stream including the encoded side information to obtain side information, and may send the side information to the geometric information reconstruction module 206 of the point cloud. The specific decoding method will be described in the following embodiments.

The geometric information reconstruction module 206 of the point cloud may reconstruct the geometric of the current point cloud frame according to the received depth map information of the reconstructed current point cloud frame sent by the image or video based decoding module 202, the occupancy map information of the reconstructed current point cloud frame sent by the point cloud occupancy map filling module 205, and the auxiliary information sent by the auxiliary information decoding module 204, to obtain a reconstructed point cloud. The reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process may refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100, which is not described herein. The geometrical information reconstruction module 206 of the point cloud may also send the reconstructed point cloud to the point cloud filtering module 207. The point cloud filtering module 207 may filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smoothed reconstructed point cloud, and the specific filtering process may refer to the filtering process of the point cloud filtering module 114 in the encoder 100, which is not described herein. The point cloud filtering module 207 may send the smoothed reconstructed point cloud to the texture information reconstruction module 208 of the point cloud. The texture information reconstruction module 208 of the point cloud may reconstruct the texture information of the reconstructed point cloud after receiving the smoothed reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video based decoding module 202, to obtain a reconstructed point cloud reconstructed by the texture information.

It should be understood that the decoder 200 shown in fig. 6 is merely an example, and that the decoder 200 may include more or fewer modules than shown in fig. 6 in a specific implementation. This is not limiting in the embodiments of the present application.

In order to facilitate understanding, technical terms related to embodiments of the present application will be described first.

In one example, the current point cloud group may be a group of frames (GOF), and one frame group may include 32 frame point clouds, that is, the current point cloud group may include 32 frame point clouds. Of course, the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application. In one example, a group of frames may also be divided into a plurality of subgroups, each subgroup may include one or more frame point clouds, and the current point cloud group may be any one of the plurality of subgroups. For example, referring to fig. 7, fig. 7 is a schematic diagram of a frame group provided in an embodiment of the present application, as shown in fig. 7, one frame group may be divided into 10 subgroups, a first subgroup includes 1,2,3 frame point clouds, a second subgroup includes 4 frame point clouds, a third subgroup includes 5,6,7,8 frame point clouds, … …, a 10 th subgroup includes 30,31,32 frame point clouds, and the current point cloud group may be any subgroup of the 10 subgroups. The point clouds included in the first, second, third, and tenth subgroups are shown in fig. 7 by way of example only, and the point clouds included in the other subgroups are not specifically described. In summary, the current point cloud group may be a frame group, or the current point cloud group may be a subgroup in a frame group, and of course, the current point cloud group may also be another type of point cloud group, such as a custom point cloud group, which is not limited in this embodiment of the present application.

Based on the above description, the patches may be classified into three types, namely, a first type of patch, a second type of patch, and a third type of patch. Illustratively, depending on how well a patch matches within the current point cloud group, the patch may also be categorized into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, a first type of patch may be a globally matching patch, a second type of patch may be a locally matching patch, and a third type of patch may be a non-matching patch. For convenience of description, taking the to-be-encoded patch as an example, it should be understood that the to-be-encoded patch is any patch in the current point cloud frame, and the current point cloud frame is any frame point cloud in the current point cloud group. If the to-be-encoded patch is a first type of patch, the to-be-encoded patch has a patch with a matching relationship in all point cloud frames in the current point cloud group, and the to-be-encoded patch can be called a global matching patch; if the to-be-encoded patch is of a second type, the to-be-encoded patch has a patch with a matching relationship in a reference frame of the current point cloud frame, but only has a patch with a matching relationship in a part of the point cloud frames in the current point cloud group, which can be called as a local matching patch; if the to-be-encoded patch is a third type of patch, the to-be-encoded patch does not exist in any point cloud frame in the current point cloud group, and the to-be-encoded patch can be called as a non-matching patch.

In one possible implementation, referring to fig. 8, the reference frame of the current point cloud frame may be located within the current point cloud group. For example, the reference frame of the current point cloud frame may be a previous point cloud frame of the current point cloud frame. In addition, the patches included in all the point cloud frames in the current point cloud group can be sequentially arranged according to the order of the first class patches, the second class patches and the third class patches, wherein the number of the first class patches in each point cloud frame in the current point cloud group is the same, and when the number of the first class patches is greater than 1, the first class patches in each point cloud frame in the current point cloud group can be sequentially arranged according to the same order, that is, the first class patches in the same position in each point cloud frame have a matching relationship. For example, the w first type of patches in each point cloud frame have a matching relationship with each other, and w is less than or equal to the number of the first type of patches.

In general, two-dimensional information, rotation transformation information and normal coordinate axis information of the first type patch in the occupancy map of the current point cloud frame are the same, and three-dimensional information is different. However, in some cases, the two-dimensional information of the first type of patch may be different, or the three-dimensional information may be the same. That is, the rotation transformation information and normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. Thus, for two-dimensional information and three-dimensional information, there may be three cases. For convenience of description, a patch to be encoded will be described as an example. In a first possible case, when the to-be-encoded latch is a first type latch, the two-dimensional information of the reference latch of the to-be-encoded latch is identical to the two-dimensional information of the to-be-encoded latch, and the three-dimensional information of the reference latch of the to-be-encoded latch is identical to the three-dimensional information of the to-be-encoded latch. In a second possible case, when the to-be-encoded latch is the first type latch, the two-dimensional information of the reference latch of the to-be-encoded latch is the same as the two-dimensional information of the to-be-encoded latch, and the three-dimensional information of the reference latch of the to-be-encoded latch is different from the three-dimensional information of the to-be-encoded latch. In a third possible case, when the to-be-encoded latch is the first type latch, the two-dimensional information of the reference latch of the to-be-encoded latch is different from the two-dimensional information of the to-be-encoded latch, and the three-dimensional information of the reference latch of the to-be-encoded latch is different from the three-dimensional information of the to-be-encoded latch.

When the patches included in all the point cloud frames in the current point cloud group are arranged in the manner shown in fig. 8, the reference patch of the patch to be encoded may be a patch in the same position in the reference frame of the current point cloud frame where the patch to be encoded is located. For example, the to-be-encoded latch is located at the third position in the current point cloud frame, and then the third latch in the reference frame of the current point cloud frame is the reference latch of the to-be-encoded latch.

The rotation conversion information may be rotation angle information, coordinate axis conversion information, or mirror image conversion information. Wherein the rotation angle information includes a specific rotation angle value or rotation angle index. If the rotation angle value of the patch to be encoded is 0, the patch to be encoded is not rotated; if the rotation angle value of the patch to be encoded is 20, the patch to be encoded is rotated by 20 degrees; if the rotation angle value of the patch to be encoded is 80, the patch to be encoded is rotated by 80 degrees. And if the rotation angle index of the patch to be encoded is 0, the patch to be encoded is not rotated; if the rotation angle index of the patch to be encoded is 1, the patch to be encoded is rotated by 15 degrees; if the rotation angle index of the patch to be encoded is 2, the patch to be encoded is rotated by 30 degrees; if the rotation angle index of the patch to be encoded is 3, it indicates that the patch to be encoded rotates 45 °. The rotation angle information may be characterized in other forms, and is not particularly limited herein. For example, the description may be made in a matrix form, and particularly, when the acceleration processing is performed in the GPU (Graphic Procession Unit, image processor), the matrix form is used to increase the processing speed.

It should be noted that the rotation of the patch to be encoded may be either clockwise or counterclockwise.

If the rotation transformation information is coordinate axis exchange information, the coordinate axis exchange information may be represented as a coordinate axis exchange identifier, for example, when the coordinate axis exchange identifier is false or 0, it is represented that the tangential coordinate axis of the patch to be encoded and the double-tangential coordinate axis of the patch to be encoded are not exchanged; for example, when the coordinate axis exchange identifier is wire or 1, the tangential coordinate axis of the patch to be encoded and the double tangential coordinate axis of the patch to be encoded are exchanged. For example, the index of the coordinate axis of the patch to be encoded may be 0,1, or 2. Assuming that the index of the X axis is 0, the index of the y axis is 1, and the index of the z axis is 2, the tangential coordinate axis index of the patch to be encoded and the double tangential coordinate axis index of the patch to be encoded may be any 2 of 0,1, and 2. In general, the tangential axis of a patch may be referred to as the U-axis and the tangential axis of a patch may be referred to as the V-axis. Assuming that in a certain situation, the obtained tangential coordinate axis index of the current patch and the obtained double-tangential coordinate axis index of the patch to be encoded are respectively 0 and 2, if the coordinate axis exchange identifier is false or 0, the index of the tangential coordinate axis of the patch to be encoded and the index of the double-tangential coordinate axis of the patch to be encoded are not exchanged; if the coordinate axis exchange identifier is wire or 1, exchanging the index of the tangential coordinate axis of the patch to be encoded and the index of the double-tangential coordinate axis of the patch to be encoded, namely updating the index of the tangential coordinate axis of the patch to be encoded to 2, and updating the index of the double-tangential coordinate axis of the patch to be encoded to 0.

Next, a first point cloud encoding method provided in the embodiments of the present application is described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the following point cloud encoding methods may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100.

Fig. 9 is a flowchart of a point cloud encoding method according to an embodiment of the present application, where the method is applied to a point cloud decoding system. Referring to fig. 9, the method includes:

s901: and acquiring auxiliary information of the patch to be encoded and a first identification of the patch to be encoded.

S902: when the first mark is a first value, the auxiliary information of the patch to be encoded and the first mark are encoded into a code stream, and the patch to be encoded is the first patch in the current point cloud frame; or when the first mark is a first value, the auxiliary information of the patch to be encoded is encoded into the code stream, and the patch to be encoded is the non-first patch in the current point cloud frame.

In general, since the first type of patch has a patch with a matching relationship in all the point cloud frames in the current point cloud group, two-dimensional information, rotation transformation information and normal coordinate axis information between the first type of patches in the current point cloud group are the same, and three-dimensional information may not be the same. Therefore, in order to reduce the bit overhead of the code stream and further improve the coding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be coded may include three-dimensional information, but not include two-dimensional information, rotation transformation information and normal coordinate axis information.

In one possible scenario, the patches in each point cloud frame may be arranged in the order of the first type of patch, the second type of patch, and the third type of patch, and then encoded in the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, if the first type of patch exists in the current point cloud group, the first patch in each point cloud frame is the first type of patch. The number of the first class of patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first class of patches is greater than 1, the first class of patches in each point cloud frame in the current point cloud group can be sequentially arranged according to the same sequence, that is, the first class of patches in the same position in each point cloud frame have a matching relationship. If the first type of patch does not exist in the current point cloud group, the first patch in each point cloud frame may be the second type of patch or the third type of patch. If the first patch in each point cloud frame is a third type of patch, the current point cloud group can only include the one point cloud frame, and the first identifications of all patches in the point cloud frame are third values.

For the case that the first latch in each point cloud frame in the current point cloud group is the first type latch or the second type latch, in order to facilitate distinguishing which point cloud frames can form a point cloud group, as shown in fig. 10, for the first point cloud frame in the current point cloud group, the first identifier of all latches included in the first point cloud frame can be set to a third value, and the latches in other point cloud frames are arranged according to the order of the first type latch, the second type latch and the third type latch. Thus, when the first identifier of the patch to be encoded is a first value, it may be determined that the current point cloud frame is a non-first point cloud frame within the current point cloud group. Moreover, the current point cloud frame may be a kth point cloud frame in the current point cloud group, where K is an integer greater than or equal to 3. That is, for the 3 rd point cloud frame and each point cloud frame after the 3 rd point cloud frame in the current point cloud group, the encoding may be performed in the manner of S902.

Based on the above description, the 3 rd point cloud frame and each point cloud frame after the 3 rd point cloud frame in the current point cloud group may be encoded in the manner of S902, and the 2 nd point cloud frame in the current point cloud group may be encoded in the following two manners:

As an example, a partial syntax structure when the first identifier of the patch to be encoded is a first value is illustrated herein, see tables 1 and 2.

TABLE 1

TABLE 2

The skip_patch_data_unit in table 1 is information of the to-be-encoded patch encoded in the bitstream when the first identifier of the to-be-encoded patch is the first value, and may also be referred to as a data unit of the to-be-encoded patch.

When the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier can be encoded into the code stream. That is, for each patch in the first point cloud frame within the current point cloud group, the auxiliary information and the first identification for each patch may be encoded into the codestream. For non-first type patches in non-first point cloud frames in the current point cloud group, auxiliary information and a first identifier of each patch can be encoded into a code stream.

In general, the second class of latches has a latch with a matching relationship in a reference frame of the current point cloud frame, rotation transformation information and normal coordinate axis information between the second class of latches and the reference latch are the same, and two-dimensional information and three-dimensional information may not be the same, and the third class of latches does not have a latch with a matching relationship in a reference frame of the current point cloud frame. When the first flag is a second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of a reference patch of the patch to be encoded, without including rotation transformation information and normal coordinate axis information. When the first flag is a third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

In a second possible implementation manner, the first identifier of the first latch in the current point cloud frame and the first identifier of the latch with the value of the second value or the third value outside the first latch may be respectively packaged and encoded into a code stream, and auxiliary information of each latch included in the current point cloud frame may also be respectively packaged and encoded into a code stream. For example, a first identification of a first patch in the current point cloud frame may be encoded into the code stream, and then auxiliary information of the first patch may be encoded into the code stream. And then, encoding auxiliary information of other first-class patches in the current point cloud frame into the code stream one by one. Finally, the remaining patches, i.e., the second class of patch and the third class of patch, may be encoded in a manner of encoding the first identifier first and then encoding the auxiliary information.

In a third possible implementation manner, the first identifiers of the plurality of latches included in the current point cloud frame may be packaged together and encoded into the code stream, where the first identifiers of the plurality of latches include the first identifiers of the latches to be encoded, and the remaining first identifiers of the latches of the plurality of latches that are valued as the second value or the third value except the latches to be encoded. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in each patch auxiliary information can be packaged together and encoded into a code stream. Different types of information in the side information for each patch may be individually packetized and encoded into the codestream.

In order to further reduce the bit overhead of the code stream, when the coding is performed according to the sequence shown in fig. 8, coding may be performed by other modes than the foregoing coding mode for the third class of latches, for example, if the to-be-coded latch is the first third class of latches, then the coding may be performed by a fixed-length coding mode or a variable-length coding mode, which is not limited in the embodiment of the present application. If the patch to be encoded is not the first third type of patch, at this time, the two-dimensional information and the three-dimensional information of the patch to be encoded can be encoded into the code stream by adopting a differential encoding mode according to the two-dimensional information and the three-dimensional information of the first third type of patch, and other information can be encoded by adopting the mode described above. According to the two-dimensional information and the three-dimensional information of the first third type of patch, encoding the two-dimensional information and the three-dimensional information of the patch to be encoded into the code stream by adopting a differential encoding mode means that: and encoding a difference value between the two-dimensional information of the patch to be encoded and the two-dimensional information of the first third type of patch and a difference value between the three-dimensional information of the patch to be encoded and the three-dimensional information of the first third type of patch into a code stream.

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are acquired first. Because the number of the first type of patches in the current point cloud group in each point cloud frame can be the same, when the first mark is a first value and the patch to be encoded is the first patch in the current point cloud frame, the condition that the patch to be encoded is the first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information and the first mark of the patch to be encoded can be encoded into a code stream; when the first mark is a first value and the to-be-encoded patch is the non-first patch in the current point cloud frame, the to-be-encoded patch is indicated to be the non-first type patch in the current point cloud frame, and at the moment, the auxiliary information of the to-be-encoded patch can be encoded into the code stream without encoding the first mark of the to-be-encoded patch into the code stream, so that the format of the encoded code stream can be simplified, the bit cost of the code stream can be further reduced, and the encoding efficiency can be improved.

Next, a point cloud decoding method corresponding to the point cloud encoding method according to the embodiment shown in fig. 9 will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the following point cloud decoding methods may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 11, the decoding method includes:

S1101: when the to-be-decoded patch is the first patch in the current point cloud frame, analyzing a first identification of the to-be-decoded patch from the code stream.

S1102: and when the first mark is a first value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded.

As can be seen from the embodiment shown in fig. 9, when the first flag is the first value, it may indicate that the current point cloud frame is a non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the patch to be decoded may be decoded by the following two possible implementations.

In a first possible implementation manner, the first identifier of the first latch in the 2 nd point cloud frame in the current point cloud group and the number of the first type latches included in each point cloud frame in the current point cloud group are encoded into the code stream together. At this time, if the current point cloud frame is the 2 nd point cloud frame in the current point cloud group, the to-be-decoded latch is the first latch in the current point cloud frame, the first identifier of the to-be-decoded latch may be first parsed from the code stream, and when the first identifier is the first value, the number of the first type latches in the current point cloud group is parsed from the code stream. And then analyzing auxiliary information of the patch to be decoded from the code stream.

Next, a case where the to-be-decoded latch is the nth latch in the current point cloud frame will be described through S1103 and S1104.

S1103: when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises three-dimensional information of the to-be-decoded patch.

As can be seen from the embodiment shown in fig. 9, in the encoding process of the first type of patch, the first identifier of the non-first type of patch is not encoded into the code stream, so when the to-be-decoded patch is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of the first type of patches in the reference point cloud frame of the current point cloud frame, it indicates that the to-be-decoded patch is the non-first type of patch in the current point cloud frame, and at this time, the auxiliary information of the to-be-decoded patch can be parsed from the code stream, where the auxiliary information includes the three-dimensional information of the to-be-decoded patch.

When the patch in each point cloud frame in the current point cloud group is encoded according to the arrangement manner shown in fig. 8, the reference patch of the patch to be decoded may be a patch whose position in the reference frame of the current point cloud frame is the same as that of the patch to be decoded in the current point cloud frame.

The method for analyzing the three-dimensional information of the patch to be decoded from the code stream by adopting the differential decoding method can be specifically as follows: the three-dimensional information of the patch with the same position as the patch to be decoded in the current point cloud frame is obtained from the decoded point cloud, or the three-dimensional information of the reference patch of the patch to be decoded is obtained from the reference frame of the current point cloud frame. And then, adding the obtained three-dimensional information with the three-dimensional information difference value analyzed from the code stream, thereby obtaining the three-dimensional information of the patch to be decoded.

Based on the description of S1102, the number of the first class patches included in each point cloud frame in the current point cloud group may be determined in two ways. In a first possible implementation manner, after decoding the first identifier of the first latch in the 2 nd point cloud frame in the current point cloud group, the number of the first type latches in the current point cloud group can be directly resolved from the code stream. In a second possible implementation manner, after decoding the first identifier of all the patches of the 2 nd point cloud frame in the current point cloud group, the number of patches with the first identifier as the first value may be determined, so that the number of first class patches included in the 2 nd point cloud frame, that is, the number of first class patches in the current point cloud group, may be determined. That is, the number of the first class patches included in each point cloud frame in the current point cloud group can be determined through the 2 nd point cloud frame.

In one possible scenario, the 2 nd point cloud frame within the current point cloud group may be the reference point cloud frame of the 3 rd point cloud frame, the 3 rd point cloud frame may be the reference point cloud frame of the 4 th point cloud frame, and so on. In this way, the 3 rd point cloud frame and the point cloud frames after the 3 rd point cloud frame can determine whether the nth patch in the current point cloud frame is the first type patch according to the number of the first type patches in the reference point cloud frame.

S1104: when the to-be-decoded patch is the nth patch in the current point cloud frame, wherein n is greater than 1 and greater than the number of the first class patches in the reference point cloud frame of the current point cloud frame, the first identification of the to-be-decoded patch is analyzed from the code stream.

S1105: and when the first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch.

Since the auxiliary information of the second type of patch, which is encoded into the code stream in the encoding process, comprises two-dimensional information, three-dimensional information and identification information of the reference patch, but does not comprise rotation transformation information and normal coordinate axis information, when the to-be-decoded patch is the second type of patch, the auxiliary information analyzed from the code stream comprises the two-dimensional information, the three-dimensional information of the to-be-decoded patch and the identification information of the reference patch of the to-be-decoded patch, but does not comprise the rotation transformation information and the normal coordinate axis information of the to-be-decoded patch. At this time, rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.

The identification information of the reference patch of the patch to be decoded may be parsed from the code stream by a method matched with the encoding end, for example, may be parsed by a fixed-length decoding method. For the rotation transformation information and normal coordinate axis information of the to-be-decoded latch, the reference latch of the to-be-decoded latch can be determined through the identification information of the reference latch of the to-be-decoded latch, so that the rotation transformation information and the normal coordinate axis information of the reference latch of the to-be-decoded latch are obtained, and the rotation transformation information and the normal coordinate axis information of the reference latch are used as the rotation transformation information and the normal coordinate axis information of the to-be-decoded latch. For the two-dimensional information and the three-dimensional information of the patch to be decoded, as an example, a method matched with the encoding end can be adopted for decoding, for example, the two-dimensional information and the three-dimensional information of the patch to be decoded can be resolved from the code stream in a differential decoding mode. Specifically, the reference patch of the patch to be decoded can be determined through the identification information of the reference patch of the patch to be decoded, so that the two-dimensional information and the three-dimensional information of the reference patch of the patch to be decoded are obtained. And then, correspondingly adding the acquired two-dimensional information and three-dimensional information with the two-dimensional information difference value and the three-dimensional information difference value which are analyzed from the code stream respectively, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.

S1106: and when the first mark is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

Based on the above description, the first class patch, the second class patch and the third class patch in each point cloud frame in the current point cloud group may be arranged as shown in fig. 8, where if the patch to be decoded is the first patch in the current point cloud frame, after the first identifier of the patch to be decoded is parsed from the code stream, it may be determined whether the current point cloud frame is the first frame point cloud in the current point cloud group according to the value of the first identifier. That is, when the latch to be decoded is the first latch in the current point cloud frame and the first identifier is the third value, the current point cloud frame may be determined to be the first frame point cloud in the current point cloud group.

As an example, the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded may be resolved from the code stream by adopting a fixed-length decoding manner, or the two-dimensional information, the three-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded may be resolved from the code stream by adopting a variable-length decoding manner. However, if the auxiliary information of the first third type patch is encoded according to the fixed-length encoding method or the variable-length encoding method, the two-dimensional information and the three-dimensional information of the non-first third type patch are encoded by the differential encoding method, and the other auxiliary information is encoded by the fixed-length encoding method. In this step, for the non-first third type of patch, the two-dimensional information and the three-dimensional information of the patch to be decoded can be resolved from the code stream by adopting a differential decoding mode, and the rotation transformation information and the normal coordinate axis information of the patch to be decoded can be resolved from the code stream by adopting a fixed-length decoding mode. For the non-first third type of patch, the method for analyzing the two-dimensional information and the three-dimensional information of the patch to be decoded from the code stream by adopting the differential decoding method is as follows: analyzing the difference value between the two-dimensional information of the patch to be decoded and the two-dimensional information of the first third class of patch from the code stream, and adding the difference value and the two-dimensional information of the first third class of patch to obtain the two-dimensional information of the patch to be decoded. Analyzing the difference value between the three-dimensional information of the patch to be decoded and the three-dimensional information of the first third class of patch from the code stream, and adding the difference value and the three-dimensional information of the first third class of patch to obtain the three-dimensional information of the patch to be decoded.

S1107: reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

In this embodiment of the present application, since the first identifier of the first latch in the current point cloud frame is encoded into the code stream, when the point cloud block latch to be decoded is the first latch in the current point cloud frame, the first identifier of the latch to be decoded may be parsed from the code stream, and when the first identifier is the first value, it indicates that the latch to be decoded is the first type latch, at this time, auxiliary information of the latch to be decoded may be parsed from the code stream, where the auxiliary information includes three-dimensional information of the latch to be decoded. When the to-be-decoded latch is the nth latch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first latches in the reference point cloud frame of the current point cloud frame, indicating that the to-be-decoded latch is the non-first latch, at this time, the auxiliary information of the to-be-decoded latch can be analyzed from the code stream, and the auxiliary information comprises the three-dimensional information of the to-be-decoded latch. When the first identifier is a second value, indicating that the to-be-decoded patch is a second type patch, at this time, auxiliary information of the to-be-decoded patch can be analyzed from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch. When the first identifier is a third value, the to-be-decoded latch is indicated to be a third type latch, and at the moment, auxiliary information of the to-be-decoded latch can be analyzed from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the to-be-decoded latch. That is, for the first class of patch, the first identifier needs to be resolved from the code stream, and for the non-first class of patch, the first identifier does not need to be resolved from the code stream, and for the first class of patch and the second class of patch, only part of auxiliary information needs to be resolved from the code stream, and other auxiliary information can be derived or obtained, so that the format of resolving the code stream can be simplified, the bit overhead of the code stream is reduced, and the decoding efficiency is improved.

The second point cloud encoding method provided in the embodiment of the present application is described below. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the following point cloud encoding methods may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100. Referring to fig. 12, the method includes:

s1201: and acquiring auxiliary information of the patch to be encoded and a first identification of the patch to be encoded.

In the case of description, the content of S1201 may refer to S901 in the embodiment shown in fig. 9, which is not described in detail in this embodiment of the present application.

S1202: when the first mark is a first value, encoding auxiliary information of the patch to be encoded, the first mark and a second mark of the patch to be encoded into a code stream, wherein the patch to be encoded is the first patch in the current point cloud frame; or when the first identifier is a first value, the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is a non-first patch in the current point cloud frame.

In general, the first type of patch has patches with matching relation in all the point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis information among the first type of patches in the current point cloud group are the same, but the three-dimensional information may not be the same, but in some cases, the three-dimensional information of the first type of patches may also be the same. In this case, when the first flag is a first value, the patch to be encoded may be indicated as a first type of patch, and the information encoded into the code stream may further include a second flag.

As an example, a partial syntax structure when a first identifier of a patch to be encoded is a first value and information encoded into a bitstream includes a second identifier is illustrated herein, see tables 1 and 3.

TABLE 3 Table 3

Further, in one possible case, when the to-be-encoded patch is the first value, that is, when the to-be-encoded patch is the first type patch, the three-dimensional information of the to-be-encoded patch is different from the three-dimensional information of the reference patch, and the two-dimensional information of the to-be-encoded patch is different from the two-dimensional information of the reference patch. Thus, when the first flag is a first value, the information encoded into the code stream may include, in addition to the second flag, a third flag indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the third flag may be override_3d_shift_data_flag. The second identifier is already described, so that no further description is given here.

When the third flag is a sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so that the two-dimensional information of the patch to be encoded does not need to be encoded into the code stream again, that is, the auxiliary information of the patch to be encoded may not include the two-dimensional information. When the third identifier is a seventh value, it may indicate that the two-dimensional information of the patch to be encoded is already encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch of the patch to be encoded, so that the two-dimensional information of the patch to be encoded needs to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include the two-dimensional information. The syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.

As an example, a partial syntax structure when a first identifier of a patch to be encoded is a first value and information encoded into a bitstream includes a second identifier and a third identifier is illustrated herein, see tables 1 and 4.

TABLE 4 Table 4

When the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch, the information of the coded stream can be indicated to include the two-dimensional information and the three-dimensional information of the patch to be encoded by the second identifier and the third identifier. It may also be indicated in other manners, for example, whether the two-dimensional information and the three-dimensional information of the patch to be encoded have been encoded into the code stream may also be indicated by the fourth identifier, that is, when the first identifier is the first value, the patch to be encoded may be indicated as the first type of patch, and the information encoded into the code stream may further include the fourth identifier. The fourth identified syntax element may be an override_latch_data_flag. When the fourth flag is the eighth value, it may be indicated that the two-dimensional information and the three-dimensional information of the patch to be encoded are both the same as the two-dimensional information and the three-dimensional information of the reference patch of the patch to be encoded, and the code stream is not encoded. When the fourth flag is the ninth value, it may be indicated that the two-dimensional information and the three-dimensional information of the patch to be encoded are different from the two-dimensional information and the three-dimensional information of the reference patch of the patch to be encoded, and the two-dimensional information and the three-dimensional information are both encoded into the code stream. The syntax element of the eighth value may be a flag, and the syntax element of the ninth value may be true.

Wherein, the three-dimensional information of the patch to be encoded may include an offset (3d_shift_agent_axis) of the patch to be encoded in the 3D space along a tangential direction, an offset (3d_shift_bitmap_axis) of the patch to be encoded in the 3D space along a double tangential direction, and an offset (3d_shift_normal_axis) of the patch to be encoded in the 3D space along a normal direction. The two-dimensional information of the patch to be encoded may include an offset (2d_shift_u) of the patch to be encoded in the u-axis direction in the current point cloud frame occupancy map, an offset (2d_shift_v) of the patch to be encoded in the v-axis direction in the current point cloud frame occupancy map, a width (2d_size_u) of the patch to be encoded in the current point cloud frame occupancy map, and a height (2d_size_v) of the patch to be encoded in the current point cloud frame occupancy map. Therefore, it is also indicated by an identification (overlay_3d_shift_array) for indicating whether 3d_shift_array has been encoded into a code stream, an identification (overlay_2d_shift_v) for indicating whether 2d_shift_array has been encoded into a code stream, and an identification (overlay_3d_shift_array_array) for indicating whether 3d_array has been encoded into a code stream, and by an identification (overlay_2d_shift_u) for indicating whether 2d_shift_u has been encoded into a code stream, an identification (overlay_2d_shift_v) for indicating whether 2d_shift_v has been encoded into a code stream, an identification (overlay_2d_array_array_array_v) for indicating whether 2d_size has been encoded into a code stream, and whether two-dimensional information to be encoded into a code stream, whether or not the two-dimensional information to indicate whether the encoded patch has been encoded into a code stream. Similar to the second identifier, the third identifier, and the fourth identifier, each identifier may have two values, and syntax elements of the two values may be true and false. The meaning of each value may refer to the meaning of a different value of the second identifier, the third identifier or the fourth identifier, which is not described herein.

As an example, a first identifier of a patch to be encoded is exemplified herein as a first value, and information encoded into a bitstream includes a fourth identifier, and a partial syntax structure for indicating identifiers of different two-dimensional information and three-dimensional information, respectively, as shown in tables 1 and 5.

TABLE 5

/>

It should be noted that, the content of the embodiment of the present application that is the same as that of the embodiment shown in fig. 9 is not repeated, and reference may be made to the description in the embodiment shown in fig. 9.

In the embodiment of the application, the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded are acquired first. Because the number of the first type of patches in the current point cloud group in each point cloud frame can be the same, when the first mark is a first value and the patch to be encoded is the first patch in the current point cloud frame, the condition that the patch to be encoded is the first type of patch in the current point cloud frame is indicated, and at the moment, the auxiliary information, the first mark and the second mark of the patch to be encoded can be encoded into a code stream; when the first mark is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, the to-be-encoded patch is indicated to be a non-first type patch in the current point cloud frame, auxiliary information and the second mark of the to-be-encoded patch can be encoded into the code stream, and the first mark of the to-be-encoded patch is not encoded into the code stream, so that the format of the encoded code stream can be simplified. The second identifier is an identifier for indicating whether the three-dimensional information of the patch to be encoded is encoded into the code stream. When the second identifier is a fourth value, the three-dimensional information indicating the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so that the three-dimensional information of the patch to be encoded does not need to be encoded into the code stream again, that is, the auxiliary information of the patch to be encoded does not include the three-dimensional information, so that the bit overhead of the code stream can be further reduced, and the encoding efficiency is improved.

Next, a point cloud decoding method based on the point cloud encoding method of the embodiment shown in fig. 12 described above will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the following point cloud decoding methods may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 13, the decoding method includes:

s1301: when the to-be-decoded patch is the first patch in the current point cloud frame, analyzing a first identification of the to-be-decoded patch from the code stream.

Note that, the content of S1301 may refer to the content of S1101, which is not described herein.

S1302: and when the first identifier is a first value, analyzing a second identifier of the patch to be decoded from the code stream.

Note that, the second identifier is already described in S1202, so that a description thereof is omitted here.

S1303: when the second mark is a fourth value, determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; and when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded.

In some embodiments, based on the description of step 1202, in a case where the three-dimensional information of the pad to be encoded is different from the three-dimensional information of the reference pad, and the two-dimensional information of the pad to be encoded is different from the two-dimensional information of the reference pad, the information encoded into the code stream may further include a third identifier for indicating whether the two-dimensional information of the pad to be encoded has been encoded into the code stream, in addition to the second identifier. That is, when the first identifier is the first value, the third identifier of the patch to be decoded may be parsed from the code stream in addition to the second identifier of the patch to be decoded. When the third flag is a sixth value, it may indicate that the two-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the two-dimensional information of the reference pad of the patch to be decoded is the same as the two-dimensional information of the reference pad of the patch to be decoded, so the two-dimensional information of the reference pad of the patch to be decoded can be used as the two-dimensional information of the patch to be decoded. When the third flag is a seventh value, it may indicate that the two-dimensional information of the patch to be decoded is already encoded into the code stream. That is, the two-dimensional information of the patch to be decoded is different from the two-dimensional information of the reference patch of the patch to be decoded, so that the two-dimensional information of the patch to be decoded can be parsed from the code stream. The specific decoding method may be a decoding method matched with the encoding method, which is not described herein.

In other embodiments, based on the description of step 1202, in the case where the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch, it is not only indicated by the second identifier and the third identifier that the information of the coded stream includes the two-dimensional information and the three-dimensional information of the patch to be decoded. It may also be indicated in other ways, for example, whether the two-dimensional information and the three-dimensional information of the patch to be decoded are both encoded in the code stream may also be indicated by the fourth flag. That is, when the first identifier is the first value, the fourth identifier of the patch to be decoded may be parsed from the code stream. When the fourth identifier is the eighth value, the two-dimensional information and the three-dimensional information of the to-be-decoded patch can be indicated to be the same as the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch, and the code stream is not encoded, so that the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch can be used as the two-dimensional information and the three-dimensional information of the to-be-decoded patch. When the fourth identifier is a ninth value, the two-dimensional information and the three-dimensional information of the to-be-decoded patch can be indicated to be different from the two-dimensional information and the three-dimensional information of the reference patch of the to-be-decoded patch and are encoded into the code stream, so that the two-dimensional information and the three-dimensional information of the to-be-decoded patch can be analyzed from the code stream. The specific decoding method may be a decoding method matched with the encoding method, which is not described herein.

And, for the three-dimensional information to be decoded, the three-dimensional information to be decoded is indicated by an identification (overlay_3d_shift_array) for indicating whether 3d_shift_tan_array has been encoded into the code stream, an identification (overlay_2d_shift_array) for indicating whether 3d_shift_array has been encoded into the code stream, an identification (overlay_2d_array) for indicating whether 3d_shift_array has been encoded into the code stream, an identification (overlay_3d_shift_array) for indicating whether 2d_array has been encoded into the code stream, and a two-dimensional information to be decoded is indicated by an identification (overlay_2d_shift_u) for indicating whether 2d_shift_u has been encoded into the code stream, an identification (overlay_2d_array_v) for indicating whether 2d_shift_v has been encoded into the code stream, an identification (overlay_2d_array_v) for indicating whether 2d_size has been encoded into the code stream, and a two-dimensional information to be decoded by a method similar to that described above.

S1304: when the to-be-decoded patch is the nth patch in the current point cloud frame, and n is more than 1 and less than or equal to the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing a second identification of the to-be-decoded patch from the code stream.

S1305: when the second mark is a fourth value, determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; and when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded.

Note that, the content of S1305 may refer to the content of S1303 described above, and will not be described herein.

S1306: reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

Note that, the content of S1306 may refer to the content of S1107, which is not described herein.

In this embodiment of the present application, since the first identifier of the first latch in the current point cloud frame is encoded into the code stream, when the to-be-decoded point cloud block latch is the first latch in the current point cloud frame, the first identifier of the to-be-decoded latch is parsed from the code stream, and when the first identifier is the first value, it indicates that the to-be-decoded latch is the first type latch, and at this time, the second identifier of the to-be-decoded latch may be parsed from the code stream. When the to-be-decoded latch is the nth latch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first latches in the reference point cloud frame of the current point cloud frame, indicating that the to-be-decoded latch is not the first latch, and analyzing the second identification of the to-be-decoded latch from the code stream. And when the second mark is a fourth value, indicating that the three-dimensional information of the patch to be decoded is not coded into the code stream. That is, the three-dimensional information of the to-be-decoded latch is the same as the three-dimensional information of the reference latch of the to-be-decoded latch, and at this time, the three-dimensional information of the reference latch of the to-be-decoded latch can be used as the three-dimensional information of the to-be-decoded latch, so that the three-dimensional information of the to-be-decoded latch does not need to be parsed from the code stream, the format of the parsed code stream can be simplified, and the bit overhead of the code stream can be reduced, thereby improving the decoding efficiency.

The third point cloud encoding method provided in the embodiment of the present application is described below. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the encoder 100 shown in fig. 2, any of the following point cloud encoding methods may be performed by the encoder 100 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information encoding module 109 in the encoder 100. Referring to fig. 14, the method includes:

s1401: and acquiring auxiliary information of the patch to be encoded and a first identification of the patch to be encoded.

It should be noted that, the content of S1401 may refer to S901 in the embodiment shown in fig. 9, which is not described in detail in this embodiment of the present application.

S1402: and when the first identifier is a first value, encoding the first identifier of the patch to be encoded into the code stream.

S1403: and when the first identifier is a second value or a third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream.

Based on the descriptions of S1402 and S1403, it is necessary to encode the first flag into the code stream regardless of whether the first flag of the patch to be encoded is the first value, the second value, or the third value. In this case, the first identifications of all patches in the current point cloud frame may be packaged together into the codestream. Of course, the encoded streams may be individually packetized.

In the embodiment of the application, auxiliary information of a point cloud patch to be encoded and a first identification of the patch to be encoded are acquired first. Because the two-dimensional information, the rotation transformation information, the normal coordinate axis information and the three-dimensional information among the first type of the patch in the current point cloud group can be the same, when the first mark of the patch to be encoded is a first value, the patch to be encoded is indicated to be the first type of the patch, at the moment, the first mark of the patch to be encoded can be encoded into a code stream, and auxiliary information of the patch to be encoded does not need to be encoded into the code stream, so that the format of the encoded code stream can be simplified. In addition, the value of the first identifier can be a first value, a second value and a third value. Different values indicate different types of patches. Thus, different types of patches can be distinguished by the first identifier, and the content included in the auxiliary information encoded into the code stream may be different for different types of patches, and only part of the auxiliary information is encoded into the code stream for the first type of patch and the second type of patch. Thus, the format of the coded stream can be simplified, and the bit overhead of the coded stream can be further reduced, so that the coding efficiency is improved.

Next, a point cloud decoding method corresponding to the point cloud encoding method according to the embodiment shown in fig. 14 will be described. It should be noted that, in conjunction with the point cloud decoding system shown in fig. 1 and the schematic block diagram of the decoder 200 shown in fig. 6, any of the following point cloud decoding methods may be performed by the decoder 200 in the point cloud decoding system, and more specifically, may be performed by the auxiliary information decoding module 204 in the decoder 200. Referring to fig. 15, the decoding method includes:

s1501: and analyzing the first identification of the patch to be decoded from the code stream.

S1502: and when the first mark is a first value, determining auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

S1503: and when the first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch.

It should be noted that, the content of S1503 may refer to S1105 in the embodiment shown in fig. 11, which is not described herein.

S1504: and when the first mark is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information of the patch to be decoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

It should be noted that, the content of S1504 may refer to S1106 in the embodiment shown in fig. 11, which is not described herein.

S1505: reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that, the content of S1505 may refer to S1107 in the embodiment shown in fig. 11, which is not described herein.

In the embodiment of the present application, a first identifier of a point cloud block to be decoded is first parsed from a code stream, when the first identifier is a first value, the first identifier indicates that the to-be-decoded patch is a first type patch, auxiliary information of the to-be-decoded patch is not included in the code stream, and at this time, the auxiliary information of the to-be-decoded patch can be determined according to the auxiliary information of a reference patch of the to-be-decoded patch, where the auxiliary information of the to-be-decoded patch includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. When the first mark is a second value, indicating that the to-be-decoded patch is a second type patch, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the to-be-decoded patch. When the first mark is a third value, indicating that the to-be-decoded patch is a third type patch, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information of the to-be-decoded patch comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. That is, for the first class of patch, only the first identifier needs to be parsed from the code stream, other auxiliary information can be derived or obtained, and for the second class of patch, only part of auxiliary information needs to be parsed from the code stream, other auxiliary information can also be derived or obtained. Thus, the format of the analysis code stream can be simplified, and the bit cost of the code stream can be reduced, thereby improving the decoding efficiency.

Referring to fig. 16, fig. 16 is a schematic structural diagram of a first point cloud encoding apparatus according to an embodiment of the present application. As shown in fig. 16, the encoding apparatus 1600 may include:

the patch information acquisition module 1601 is configured to acquire auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch;

the auxiliary information encoding module 1602 is configured to encode, when the first identifier is a first value, auxiliary information of a patch to be encoded and the first identifier into a code stream, where the patch to be encoded is a first patch in a current point cloud frame; or (b)

The auxiliary information encoding module 1602 is configured to encode auxiliary information of a patch to be encoded into a code stream when the first identifier is a first value, where the patch to be encoded is a non-first patch in a current point cloud frame;

the first mark is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded comprises three-dimensional information.

Wherein the auxiliary information encoding module 1602 is further configured to:

Note that the patch information acquisition module 1601 is configured to perform the related content of step S901 in the embodiment shown in fig. 9, and the auxiliary information encoding module 1602 is configured to perform the related content of step S902 and step S903 in the embodiment shown in fig. 9. As an example, the patch information acquisition module 1601 may correspond to a combination of the point cloud block information generation module 101 and the packaging module 102 in fig. 2, the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the function of the patch information acquisition module 1601 may be implemented by the point cloud block information generation module 101 and the packaging module 102 in fig. 2, and the function of the auxiliary information generation module 1602 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, and the like of a patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information, identifier information, two-dimensional information, and the like of a patch to be encoded. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the encoder shown in fig. 2 above, which is not repeated here.

Referring to fig. 17, fig. 17 is a schematic structural diagram of a first point cloud decoding device according to an embodiment of the present application. As shown in fig. 17, the decoding apparatus 1700 may include:

the auxiliary information decoding module 1701 is configured to parse a first identifier of a to-be-decoded latch from a code stream when the to-be-decoded latch is a first latch in a current point cloud frame;

the auxiliary information decoding module 1701 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is a first value, where the auxiliary information includes three-dimensional information of the patch to be decoded;

the auxiliary information decoding module 1701 is further configured to parse auxiliary information of the to-be-decoded latch from the code stream when the to-be-decoded latch is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first latches in the reference point cloud frame of the current point cloud frame, and the auxiliary information includes three-dimensional information of the to-be-decoded latch;

the reconstruction module 1702 is configured to reconstruct the to-be-decoded latch according to the auxiliary information of the to-be-decoded latch.

Wherein the side information decoding module 1701 is further configured to:

Wherein, when the first flag is a first value, the side information decoding module 1701 is further configured to:

the reconstruction module 1702 is further configured to:

Wherein, when the first flag is the second value, the auxiliary information decoding module 1701 is further configured to:

the reconstruction module 1702 is further configured to:

It should be noted that, the auxiliary information decoding module 1701 is configured to perform the relevant contents of steps S1101 to S1106 in the embodiment shown in fig. 11, and the reconstruction module 1702 is configured to perform the relevant contents of step S1107 in the embodiment shown in fig. 11. As an example, the auxiliary information decoding module 1701 in fig. 17 corresponds to the auxiliary information decoding module 204 in fig. 6, and the reconstruction module 1702 in fig. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in fig. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the decoder shown in fig. 6 above, which is not repeated here.

Referring to fig. 18, fig. 18 is a schematic structural diagram of a first point cloud encoding apparatus according to an embodiment of the present application. As shown in fig. 18, the encoding apparatus 1800 may include:

the patch information obtaining module 1801 is configured to obtain auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch;

the auxiliary information encoding module 1802 is configured to encode, when the first identifier is a first value, auxiliary information of a patch to be encoded, the first identifier, and a second identifier of the patch to be encoded into a code stream, where the patch to be encoded is a first patch in a current point cloud frame; or (b)

The auxiliary information encoding module 1802 is configured to encode, when the first identifier is a first value, auxiliary information of a patch to be encoded and a second identifier of the patch to be encoded into a code stream, where the patch to be encoded is a non-first patch in a current point cloud frame;

wherein the first identifier is a first value to indicate that the patch to be encoded is a first type of patch;

the second mark is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream;

or the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded comprises the three-dimensional information.

Note that, the patch information acquiring module 1801 is used to execute the related content of step S1201 in the embodiment shown in fig. 12, and the auxiliary information encoding module 1802 is used to execute the related content of step S1202 in the embodiment shown in fig. 12. As an example, the patch information acquisition module 1801 may correspond to the combination of the point cloud block information generation module 101 and the packaging module 102 in fig. 2, the auxiliary information encoding module 1802 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the functions of the patch information acquisition module 1801 may be implemented by the point cloud block information generation module 101 and the packaging module 102 in fig. 2, and the functions of the auxiliary information generation module 1802 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, a second identifier, and the like of a patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information, identification information, two-dimensional information, and the like of a reference patch. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the encoder shown in fig. 2 above, which is not repeated here.

Referring to fig. 19, fig. 19 is a schematic structural diagram of a first point cloud decoding device according to an embodiment of the present application. As shown in fig. 19, the decoding apparatus 1900 may include:

the auxiliary information decoding module 1901 is configured to parse a first identifier of a to-be-decoded latch from a code stream when the to-be-decoded latch is a first latch in a current point cloud frame;

the auxiliary information decoding module 1901 is further configured to parse a second identifier of the patch to be decoded from the code stream when the first identifier is a first value;

the auxiliary information decoding module 1901 is further configured to determine auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is a fourth value; when the second mark is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded;

the auxiliary information decoding module 1901 is further configured to parse a second identifier of the latch to be decoded from the code stream when the latch to be decoded is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first latches in the reference point cloud frame of the current point cloud frame;

And a reconstruction module 1902, configured to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that, the auxiliary information decoding module 1901 is used for executing the relevant content of steps S1301-S1305 of the embodiment shown in fig. 13, and the reconstruction module 1902 is used for executing the relevant content of step S1306 of the embodiment shown in fig. 13. As an example, the auxiliary information decoding module 1901 in fig. 19 corresponds to the auxiliary information decoding module 204 in fig. 6, and the reconstruction module 1902 in fig. 19 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the auxiliary information decoding module 1901 may be implemented by the auxiliary information decoding module 204 in fig. 6, and the function of the reconstruction module 1902 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the decoder shown in fig. 6 above, which is not repeated here.

Referring to fig. 20, fig. 20 is a schematic structural diagram of a first point cloud encoding apparatus according to an embodiment of the present application. As shown in fig. 20, the encoding apparatus 2000 may include:

the patch information obtaining module 2001 is configured to obtain auxiliary information of a to-be-encoded point cloud patch and a first identifier of the to-be-encoded patch;

The auxiliary information encoding module 2002 is configured to encode a first identifier of a patch to be encoded into a code stream when the first identifier is a first value;

the auxiliary information encoding module 2002 is further configured to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the second value or the third value;

Note that the patch information acquisition module 2001 is used to execute the relevant contents of step S1401 of the embodiment shown in fig. 14, and the auxiliary information encoding module 2002 is used to execute the relevant contents of step S1402 and step S1403 of the embodiment shown in fig. 14. As an example, the patch information acquisition module 2001 may correspond to a combination of the point cloud block information generation module 101 and the packaging module 102 in fig. 2, the auxiliary information encoding module 2002 may correspond to the auxiliary information encoding module 109 in fig. 2, in other words, the functions of the patch information acquisition module 2001 may be implemented by the point cloud block information generation module 101 and the packaging module 102 in fig. 2, and the functions of the auxiliary information generation module 2002 may be implemented by the auxiliary information encoding module 109 in fig. 2. In one example, the point cloud block information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, a first identifier, and the like of a patch to be encoded, and the packing module 102 is configured to obtain rotation transformation information, identifier information, two-dimensional information, and the like of a patch to be encoded. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the encoder shown in fig. 2 above, which is not repeated here.

Referring to fig. 21, fig. 21 is a schematic structural diagram of a first point cloud decoding apparatus according to an embodiment of the present application. As shown in fig. 21, the decoding apparatus 2100 may include:

the auxiliary information decoding module 2101 is used for resolving a first identification of a point cloud block patch to be decoded from the code stream;

the auxiliary information decoding module 2101 is further configured to determine auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the first identifier is a first value, where the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

the auxiliary information decoding module 2101 is further configured to parse auxiliary information of a patch to be decoded from the code stream when the first identifier is a second value, where the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and identifier information of a reference patch of the patch to be decoded;

the auxiliary information decoding module 2101 is further configured to parse auxiliary information of a patch to be decoded from the code stream when the first identifier is a third value, where the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information;

and the reconstruction module 2102 is configured to reconstruct the to-be-decoded latch according to the auxiliary information of the to-be-decoded latch.

Wherein, when the first flag is the second value, the auxiliary information decoding module 2101 is further configured to:

the reconstruction module 2102 is further configured to:

It should be noted that the auxiliary information decoding module 2101 is used for executing the relevant contents of steps S1501 to S1504 in the embodiment shown in fig. 15, and the reconstruction module 2102 is used for executing the relevant contents of step S1506 in the embodiment shown in fig. 15. As an example, the auxiliary information decoding module 2101 in fig. 21 corresponds to the auxiliary information decoding module 204 in fig. 6, and the reconstruction module 2102 in fig. 21 corresponds to the geometric information reconstruction module 206 of the point cloud in fig. 5, in other words, the function of the auxiliary information decoding module 2101 may be implemented by the auxiliary information decoding module 204 in fig. 6, and the function of the reconstruction module 2102 may be implemented by the geometric information reconstruction module 206 of the point cloud in fig. 6. Of course, the present application is not limited thereto. For further details reference is made to the prior art or to the explanation of the principle of the decoder shown in fig. 6 above, which is not repeated here.

Fig. 22 is a schematic block diagram of one implementation of a coding device 2200 for an embodiment of the present application. The acquisition device 2200 may include, among other things, a processor 2201, a memory 2202, and a bus system 2203. The processor 2201 and the memory 2202 are connected by a bus system 2203, the memory 2202 is configured to store instructions, and the processor 2201 is configured to execute the instructions stored in the memory 2202 to perform various point cloud encoding or decoding methods described herein, particularly a method for filtering a current image block based on a block size of the current image block. To avoid repetition, a detailed description is not provided herein.

In the present embodiment, the processor 2201 may be a central processing unit (central processing unit, CPU), and the processor 2201 may also be other general purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 2202 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 2202. Memory 2202 may include code and data 22021 that are accessed by processor 2201 using bus 2203. The memory 2202 may further include an operating system 22023 and application programs 22022, the application programs 22022 including at least one program that allows the processor 2201 to perform the point cloud encoding or decoding methods described herein (and in particular the point cloud encoding or decoding methods described herein). For example, the application program 22022 may include applications 1 to N, which further include a point cloud encoding or decoding application (abbreviated as a point cloud decoding application) that performs the point cloud encoding or decoding method described in the present application.

The bus system 2203 may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus. For clarity of illustration, the various buses are labeled as bus system 2203 in the drawing.

Optionally, the encoding apparatus 2200 may also include one or more output devices, such as a display 2204. In one example, the display 2204 may be a touch sensitive display that incorporates a display with a touch sensitive unit operable to sense touch input. The display 2204 may be connected to the processor 2201 via a bus 2203.

It should be noted that, the decoding apparatus 2200 may perform the method of encoding the point cloud in the present application, and may also perform the method of decoding the point cloud in the present application.

Those of skill in the art will appreciate that the functions described in connection with the various illustrative logical blocks, modules, and algorithm steps described in connection with the disclosure herein may be implemented as hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by the various illustrative logical blocks, modules, and steps may be stored on a computer readable medium or transmitted as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol). In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementing the techniques described herein. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that the computer-readable storage medium and data storage medium do not include connections, carrier waves, signals, or other transitory media, but are actually directed to non-transitory tangible storage media. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, DVD, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, application Specific Integrated Circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combination codec. Moreover, the techniques may be fully implemented in one or more circuits or logic elements. In one example, the various illustrative logical blocks, units, modules in encoder 100 and decoder 200 may be understood as corresponding circuit devices or logic elements.

The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses including a wireless handset, an Integrated Circuit (IC), or a set of ICs (e.g., a chipset). The various components, modules, or units are described in this application to emphasize functional aspects of the devices for performing the disclosed techniques but do not necessarily require realization by different hardware units. Indeed, as described above, the various units may be combined in a codec hardware unit in combination with suitable software and/or firmware, or provided by an interoperable hardware unit (including one or more processors as described above).

The foregoing is merely illustrative of specific embodiments of the present application, and the scope of the present application is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of point cloud encoding, the method comprising:

acquiring auxiliary information of a point cloud block to be encoded and a first identification of the point cloud block to be encoded;

when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, encoding the auxiliary information of the to-be-encoded patch and the first identifier into a code stream;

when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, encoding auxiliary information of the to-be-encoded patch into a code stream;

the first mark is a first value to indicate that the patch to be encoded is a first type of patch, the auxiliary information of the patch to be encoded comprises three-dimensional information, and does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information, the three-dimensional information comprises offset of the patch to be encoded along tangential directions in a 3D space, offset of the patch to be encoded along double tangential directions in the 3D space and offset of the patch to be encoded along normal directions in the 3D space, and the first type of patch is global matching patch in a current point cloud group.

2. The method of claim 1, wherein the current point cloud frame is a kth point cloud frame in a current group, the K being an integer greater than or equal to 3.

3. The method according to claim 1 or 2, wherein after the obtaining the auxiliary information of the to-be-encoded point cloud patch and the first identifier of the to-be-encoded patch, the method further comprises:

when the first identifier is a second value or a third value, encoding the auxiliary information of the patch to be encoded and the first identifier into a code stream;

the first mark is a second value to indicate that the patch to be encoded is a second type of patch, the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information and identification information of a reference patch of the patch to be encoded, and the second type of patch is a local matching patch in the current point cloud group;

the first identifier is a third value to indicate that the patch to be encoded is a third type of patch, the auxiliary information of the patch to be encoded comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information, and the third type of patch is a non-matching patch in the current point cloud group.

4. A method according to any of claims 1-3, wherein said encoding said first identification into a codestream when said first identification is a first value comprises:

Encoding first identifiers of a plurality of patches included in the current point cloud frame into a code stream, wherein the first identifiers of the plurality of patches include the first identifiers of the patches to be encoded, and the first identifiers of the plurality of patches which are left in the first identifiers of the plurality of patches except the first identifiers of the patches to be encoded are second values or third values.

5. The method of claim 3 or 4, wherein said first value, said second value and said third value are binary 10, 11 and 00,

6. The method of any of claims 3-5, wherein when the first identification of the patch to be encoded is a third value and the patch to be encoded is a first patch of the current point cloud frames, the current point cloud frame is a first point cloud frame within a current point cloud group.

7. The method of any of claims 3-6, wherein the reference patch of the patch to be encoded is included in a reference frame of the current point cloud frame.

8. A method of point cloud decoding, the method comprising:

when a point cloud block patch to be decoded is the first patch in a current point cloud frame, analyzing a first identification of the patch to be decoded from a code stream;

When the analyzed first mark is a first value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded, and does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information, the three-dimensional information comprises offset of the patch to be encoded along tangential directions in a 3D space, offset of the patch to be encoded along double tangential directions in the 3D space and offset of the patch to be encoded along normal directions in the 3D space, and when the first mark is the first value, the first mark is used for indicating that the patch to be encoded is a first type of patch, and the first type of patch is a global matching patch in a current point cloud group;

when the to-be-decoded latch is an nth latch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of first-class latches in a reference point cloud frame of the current point cloud frame, analyzing auxiliary information of the to-be-decoded latch from the code stream, wherein the auxiliary information comprises three-dimensional information of the to-be-decoded latch and does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information;

reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

9. The method of claim 8, wherein the current point cloud frame is a kth point cloud frame in a current group, the K being an integer greater than or equal to 3.

10. The method of claim 8 or 9, wherein the method further comprises:

when the to-be-decoded patch is an nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of first types of patches in the reference point cloud frame of the current point cloud frame, analyzing a first identification of the to-be-decoded patch from the code stream;

when the analyzed first mark is a second value, analyzing auxiliary information of the to-be-decoded patch from the code stream, wherein the auxiliary information comprises two-dimensional information and three-dimensional information of the to-be-decoded patch and identification information of a reference patch of the to-be-decoded patch;

when the analyzed first mark is a third value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;

when the first identifier is a second value, the first identifier is used for indicating that the patch to be encoded is a second type patch, and the second type patch is a local matching patch in the current point cloud group; and when the first identifier is a third value, the first identifier is used for indicating that the to-be-encoded patch is a third class patch, and the third class patch is a non-matching patch in the current point cloud group.

11. The method of any one of claims 8-10, wherein when the to-be-decoded point cloud block latch is a first latch in the current point cloud frame, parsing the first identifier of the to-be-decoded latch from the code stream includes:

when the to-be-decoded patch is the first patch in the current point cloud frame, first identifiers of a plurality of patches included in the current point cloud frame are analyzed from the code stream, wherein the analyzed first identifiers comprise the first identifiers of the to-be-decoded patch, and the first identifiers of the plurality of first identifiers which are remained except the first identifiers of the to-be-decoded patch are second values or third values.

12. The method of claim 10 or 11, wherein the first value, the second value and the third value are binary 10, 11 and 00,

13. The method of any of claims 10-12, wherein when the patch to be decoded is a first patch in the current point cloud frame and the first identification is a third value, the current point cloud frame is a first point cloud frame.

14. The method according to any of claims 8-13, wherein when the parsed first identification is a first value, the method further comprises:

Acquiring two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;

reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded, including:

15. The method of any of claims 10-13, wherein when the parsed first indicator is a second value, the method further comprises:

16. The method of any of claims 10-15, wherein the reference patch of the patch to be decoded is included in a reference frame of the current point cloud frame.

17. A method of point cloud encoding, the method comprising:

when the first identifier is a first value and the to-be-encoded patch is the first patch in the current point cloud frame, encoding auxiliary information of the to-be-encoded patch, the first identifier and a second identifier of the to-be-encoded patch into a code stream;

when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame, encoding auxiliary information of the to-be-encoded patch and a second identifier of the to-be-encoded patch into a code stream;

the first mark is a first value to indicate that the to-be-encoded patch is a first type patch, and the first type patch is a global matching patch in the current point cloud group;

the second mark is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not coded into the code stream;

or, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded is encoded into the code stream, the auxiliary information of the patch to be encoded includes the three-dimensional information, and does not include two-dimensional information, rotation transformation information and normal coordinate axis information, and the three-dimensional information includes an offset of the patch to be encoded along tangential directions in a 3D space, an offset of the patch to be encoded along double tangential directions in the 3D space, and an offset of the patch to be encoded along normal directions in the 3D space.

18. A method of point cloud decoding, the method comprising:

when the analyzed first mark is a first value, analyzing a second mark of the patch to be decoded from the code stream;

when the to-be-decoded patch is the nth patch in the current point cloud frame, and the n is greater than 1 and smaller than or equal to the number of the first class patches in the reference point cloud frame of the current point cloud frame, analyzing the second identification of the to-be-decoded patch from the code stream;

when the second identification is analyzed to be a fourth value, determining auxiliary information of the to-be-decoded patch according to the auxiliary information of the reference patch of the to-be-decoded patch; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded, and does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information, and the three-dimensional information comprises offset of the patch to be encoded along tangential directions in a 3D space, offset of the patch to be encoded along double tangential directions in the 3D space and offset of the patch to be encoded along normal directions in the 3D space;

Reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded;

and when the first identifier is a first value, the first identifier is used for indicating that the patch to be encoded is a first type of patch, and the first type of patch is a global matching patch in the current point cloud group.

19. A point cloud encoding apparatus, the apparatus comprising:

the point cloud block patch information acquisition module is used for acquiring auxiliary information of a patch to be encoded and a first identifier of the patch to be encoded;

the auxiliary information coding module is used for coding the auxiliary information of the patch to be coded and the first identifier into a code stream when the first identifier is a first value and the patch to be coded is the first patch in the current point cloud frame;

the auxiliary information encoding module is configured to encode auxiliary information of the patch to be encoded into a code stream when the first identifier is a first value and the patch to be encoded is a non-first patch in the current point cloud frame;

20. The apparatus of claim 19, wherein the current point cloud frame is a kth point cloud frame in a current group, the K being an integer greater than or equal to 3.

21. The apparatus of claim 19 or 20, wherein the side information encoding module is further configured to:

22. The apparatus of any of claims 19-21, wherein the side information encoding module is further to:

23. The apparatus of claim 21 or 22, wherein said first value, said second value and said third value are binary 10, 11 and 00,

24. The apparatus of any of claims 21-23, wherein when the first identification of the patch to be encoded is a third value and the patch to be encoded is a first patch of the current point cloud frames, the current point cloud frame is a first point cloud frame within a current point cloud group.

25. The apparatus of any of claims 21-24, wherein the reference patch of the patch to be encoded is included in a reference frame of the current point cloud frame.

26. A point cloud decoding apparatus, the apparatus comprising:

the auxiliary information decoding module is used for analyzing a first identification of the to-be-decoded patch from the code stream when the to-be-decoded point cloud block patch is the first patch in the current point cloud frame;

The auxiliary information decoding module is further configured to parse auxiliary information of the patch to be decoded from the code stream when the parsed first identifier is a first value, where the auxiliary information includes three-dimensional information of the patch to be decoded, and does not include two-dimensional information, rotation transformation information and normal coordinate axis information, the three-dimensional information includes an offset of the patch to be encoded along a tangential direction in a 3D space, an offset of the patch to be encoded along a double tangential direction in the 3D space, and an offset of the patch to be encoded along a normal direction in the 3D space, and when the first identifier is the first value, the first identifier is used to indicate that the patch to be encoded is a first type of patch, and the first type of patch is a global matching patch in a current point cloud group;

the auxiliary information decoding module is further configured to parse auxiliary information of the to-be-decoded latch from the code stream when the to-be-decoded latch is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first-class latches in a reference point cloud frame of the current point cloud frame, and the auxiliary information includes three-dimensional information of the to-be-decoded latch and does not include two-dimensional information, rotation transformation information and normal coordinate axis information;

And the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.

27. The apparatus of claim 26, wherein the current point cloud frame is a kth point cloud frame in a current group, the K being an integer greater than or equal to 3.

28. The apparatus of claim 26 or 27, wherein the side information decoding module is further configured to:

29. The apparatus of any of claims 26-28, wherein the side information decoding module is further to:

30. The apparatus of claim 28 or 29, wherein said first value, said second value and said third value are binary 10, 11 and 00,

31. The apparatus of any of claims 28-30, wherein when the patch to be decoded is a first patch in the current point cloud frame and the first identification is a third value, the current point cloud frame is a first point cloud frame.

32. The apparatus of any of claims 26-31, wherein when the parsed first identification is a first value, the side information decoding module is further to:

the reconstruction module is further configured to:

33. The apparatus of any of claims 28-31, wherein when the parsed first identification is a second value, the side information decoding module is further to:

the reconstruction module is further configured to:

34. The apparatus of any of claims 28-33, wherein the reference patch of the patch to be decoded is included in a reference frame of the current point cloud frame.

35. A point cloud encoding apparatus, the apparatus comprising:

the point cloud block patch information acquisition module is used for acquiring auxiliary information of a point cloud block patch to be encoded and a first identification of the patch to be encoded;

the auxiliary information coding module is used for coding the auxiliary information of the patch to be coded, the first identifier and the second identifier of the patch to be coded into a code stream when the first identifier is a first value and the patch to be coded is the first patch in the current point cloud frame;

the auxiliary information encoding module is configured to encode auxiliary information of the to-be-encoded patch and a second identifier of the to-be-encoded patch into a code stream when the first identifier is a first value and the to-be-encoded patch is a non-first patch in the current point cloud frame;

36. A point cloud decoding apparatus, the apparatus comprising:

the auxiliary information decoding module is further configured to parse a second identifier of the patch to be decoded from the code stream when the parsed first identifier is a first value;

the auxiliary information decoding module is further configured to parse a second identifier of the to-be-decoded latch from the code stream when the to-be-decoded latch is an nth latch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of first latches in a reference point cloud frame of the current point cloud frame;

the auxiliary information decoding module is further configured to determine auxiliary information of the to-be-decoded latch according to auxiliary information of the reference latch of the to-be-decoded latch when the parsed second identifier is a fourth value; when the second identifier is a fifth value, analyzing auxiliary information of the patch to be decoded from the code stream, wherein the auxiliary information comprises three-dimensional information of the patch to be decoded, and does not comprise two-dimensional information, rotation transformation information and normal coordinate axis information, and the three-dimensional information comprises offset of the patch to be encoded along tangential directions in a 3D space, offset of the patch to be encoded along double tangential directions in the 3D space and offset of the patch to be encoded along normal directions in the 3D space;

The reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded;

37. A computer readable storage medium comprising program code which, when run on a computer, causes the computer to perform the point cloud encoding method of any of claims 1 to 7, or claim 17.

38. A computer readable storage medium comprising program code which, when run on a computer, causes the computer to perform the point cloud decoding method of any of claims 8 to 16, or claim 18.