CN111479114B

CN111479114B - Point cloud encoding and decoding method and device

Info

Publication number: CN111479114B
Application number: CN201910070741.XA
Authority: CN
Inventors: 张德军; 蔡康颖; 刘波
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-01-23
Filing date: 2019-01-23
Publication date: 2022-07-22
Anticipated expiration: 2039-01-23
Also published as: CN111479114A; WO2020151496A1

Abstract

The application discloses a point cloud coding and decoding method and a coder-decoder, relates to the technical field of coding and decoding, and is beneficial to improving the coding and decoding efficiency of point cloud data and reducing the bit overhead of a code stream. The method for encoding the point cloud comprises the following steps: acquiring side information of a patch to be coded; if the patch to be coded is the patch in the first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index; the global matching patch identifier is used for indicating whether the to-be-coded patch is a global matching patch in the subgroup or not; if the patch to be coded is the patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into the code stream, wherein the side information comprises three-dimensional information and a matching patch index.

Description

Point cloud encoding and decoding method and device

Technical Field

The present application relates to the field of encoding and decoding technologies, and in particular, to a point cloud (point cloud) encoding and decoding method and apparatus.

Background

With the continuous development of 3d sensor (e.g., 3d scanner) technology, it is more and more convenient to acquire point cloud data, and the scale of the acquired point cloud data is also larger and larger, so how to effectively encode the point cloud data becomes a problem that needs to be solved urgently.

After dividing the multi-frame point cloud in the point cloud group into a plurality of subgroups, if the point cloud blocks (patches) of two adjacent subgroups have a matching relationship, and the matching relationship is also used for constraining the consistency of information such as the rotation angle and the normal coordinate axis of the matching patches, the point cloud blocks (patches) in the subgroups are constrained to be subjected to certain constraint when being packed, so that the sizes of the generated depth map and the texture map are increased. And even if partial side information of the global matching patch of each frame point cloud in the subgroup is the same, all the side information of the global matching patch of each frame point cloud is coded into the code stream during coding, so that the bit cost of the code stream is increased.

Disclosure of Invention

The embodiment of the application provides a point cloud encoding and decoding method and device, which are beneficial to enabling the size of a generated depth map and a generated texture map to be smaller, reducing the bit cost of a code stream and further improving the encoding and decoding efficiency.

In a first aspect, an embodiment of the present application provides a method for encoding a point cloud, including:

acquiring side information of a patch to be coded; if the patch to be coded is the patch in the first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index; the global matching patch identifier is used for indicating whether the patch to be coded is the global matching patch in the subgroup or not; if the patch to be coded is the patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into the code stream, wherein the side information comprises three-dimensional information and a matching patch index.

As can be seen from the above, for the patch in the non-first frame point cloud in the subgroup, only part of the side information is coded into the code stream, so that the bit overhead of the code stream is reduced, and the coding efficiency is further improved.

In addition, it should be understood that the global matching patch identification of the patch (e.g., patch A) in the first frame point cloud is encoded into the codestream, in order to analyze the global matching patch identification of the patch A at the decoding end, once the patch A is used as the reference patch of other patches (for example, the patch B in the non-first frame point cloud), the global matching patch identification of the patch B can be determined based on the global matching patch identification of the patch A, further if the global matching patch of patch B is identified as a first value (e.g. l or tune), the two-dimensional information, the normal coordinate axis information, and the rotational transform information of patch B may be derived or determined from the two-dimensional information, the normal coordinate axis information, and the rotational transform information of the reference patch of patch B (i.e., patch a), therefore, the information is prevented from being coded in the code stream during coding, so that the bit overhead of the code stream is reduced, and the coding efficiency is improved.

The first frame point cloud of the subgroup refers to the 0 th frame point cloud in the subgroup, the non-first frame point cloud of the subgroup refers to the ith frame point cloud in the subgroup, i is 1,2, …, N-1, and N is the number of frames of the point clouds in the subgroup.

It should be noted that the global match patch flag of the patch to be encoded may be a first value (such as tune or 1) or a second value (such as false or 0). The first value is used for indicating that the to-be-coded patch is the global matching patch in the subgroup, and the second value is used for indicating that the to-be-coded patch is not the global matching patch in the subgroup.

The fact that the to-be-encoded patch is a global matching patch in the subset means that a patch (hereinafter referred to as a reference patch) matching the to-be-encoded patch can be found in all point clouds (other point cloud frames) except the point cloud (also referred to as a frame of point cloud or a point cloud frame) to which the to-be-encoded patch belongs in the subset. The fact that the to-be-encoded patch is not a global matching patch in the subgroup means that the to-be-encoded patch can find a patch which is matched with the to-be-encoded patch in a part of point clouds (namely some point cloud frames) in the subgroup except for the point cloud to which the to-be-encoded patch belongs, or the to-be-encoded patch cannot be found in the point clouds in the subgroup except for the point cloud to which the to-be-encoded patch belongs. Global match patches that are not within a subgroup may also be referred to as non-global match patches.

The matching patch index of the patch to be encoded is an index of a patch (hereinafter, referred to as a reference patch) that matches the patch to be encoded in the reference frame of the point cloud to which the patch to be encoded belongs. The matching patch index of the patch to be encoded may be a non-default value or a default value. The to-be-matched patch index is a non-default value to indicate a reference patch matched with the to-be-coded patch, in other words, the to-be-coded patch is a matched patch. E.g., patch [ k ]. matchedpatcIndex does not equal-1, -1 is a default value, indicating the reference patch of the patch to be encoded. The to-be-matched patch index is a default value to indicate that the to-be-coded patch has no patch matched with the to-be-coded patch, in other words, the to-be-coded patch is a non-matched patch. E.g., patch [ k ]. matchedPatchIndex equals-1, -1 is a default value indicating that the patch to be encoded does not have a patch matching it. The patch in the reference frame indicated by the matching patch index of the patch to be encoded is the reference patch.

The normal coordinate axis information may also be referred to as normal axis information, such as normalAxis, i.e. the normal axis index of the patch to be encoded.

The rotation transformation information of the patch to be encoded may be rotation angle information, coordinate axis transformation information, or mirror transformation information. The rotation angle information of the patch to be encoded includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be encoded is 0, it indicates that the patch is not rotated; if the rotation angle value of the patch to be coded is 20, the patch is rotated by 20 degrees; if the rotation angle value of the patch to be encoded is 80, it indicates that the patch is rotated by 80 °. If the rotation angle index of the patch to be coded is 0, the patch is not rotated; if the rotation angle index of the patch to be coded is 1, indicating that the patch rotates by 15 degrees; if the rotation angle index of the patch to be coded is 2, the patch is rotated by 30 degrees; if the rotation angle index of the patch to be encoded is 3, it indicates that the patch is rotated by 45 °.

It should be noted that the rotation of the to-be-encoded patch may be clockwise rotation or counterclockwise rotation, which is not limited in this application.

The coordinate axis transformation information of the to-be-encoded patch can be represented as a coordinate axis transformation index, for example, when the coordinate axis exchange identifier of the to-be-encoded patch is 0 or false, it represents that the tangent coordinate axis and the double tangent coordinate axis of the to-be-encoded patch are not exchanged; for example, when the coordinate axis exchange identifier of the patch to be encoded is 1 or tune, it indicates that the tangent coordinate axis and the bi-tangent coordinate axis of the patch to be encoded are exchanged. For example, assuming that the U-axis index is 0 and the V-axis index is 1, the index of the tangent coordinate axis of the patch to be encoded is 0 by default, which means that the tangent coordinate axis of the patch to be encoded is the U-axis; the index of the two tangent coordinate axes of the patch to be encoded is 1, which indicates that the two tangent coordinate axes of the patch to be encoded is the V axis. If the exchange identification of the coordinate axes of the patch to be coded is false, the indexes of the tangent coordinate axes of the patch and the indexes of the double tangent coordinate axes are not exchanged; if the coordinate axis exchange identification of the patch to be encoded is tune, exchanging the index of the tangent coordinate axis of the patch to be encoded and the index of the double tangent coordinate axis, that is, updating the index of the tangent coordinate axis of the patch to be encoded to 1, and updating the index of the double tangent coordinate axis of the patch to 0.

The mirror transformation information may be represented as a mirror transformation index, for example, when the mirror transformation index is 0, it represents that the U axis is transformed into the-U axis; for example, when the mirror transformation index is 1, the V axis is transformed into the-V axis; for example, when the mirror transform index is 2, it means that the V axis is transformed into the-V axis and the U axis is transformed into the-U axis.

The three-dimensional information of the to-be-encoded patch includes u1, v1 and d1, or the three-dimensional information of the to-be-encoded patch includes a difference value between u1 of the to-be-encoded patch and u1 of the reference patch thereof, a difference value between v1 of the to-be-encoded patch and v1 of the reference patch thereof, and a difference value between d1 of the to-be-encoded patch and d1 of the reference patch thereof. Where u1 (also referred to as patch _3d _ shift _ vary _ axis) is the offset of the point in the patch along the tangent axis, v1 (also referred to as patch _3d _ shift _ displacement _ axis) is the offset of the point in the patch along the bisector axis, and d1 (also referred to as patch _3d _ shift _ normal _ axis) is the offset of the point in the patch along the normal axis.

The two-dimensional information of the patch to be encoded includes u0, v0, sizeU0, and sizeU0, or the two-dimensional information of the patch to be encoded includes a difference between u0 of the patch to be encoded and u0 of its reference patch, a difference between v0 of the patch to be encoded and v0 of its reference patch, a difference between sizeU0 of the patch to be encoded and sizeU0 of its reference patch, and a difference between sizeU0 of the patch to be encoded and sizeU0 of its reference patch. Wherein u0 (also called patch _2d _ shift _ u) is an abscissa value x of the position of the occupancy map of the patch to be encoded in the point cloud occupancy map to which the occupancy map belongs, and v0 (also called patch _2d _ shift _ v) is an ordinate value y of the position of the occupancy map of the patch to be encoded in the point cloud occupancy map to which the occupancy map belongs; sizeU0 is the width of the patch to be encoded (e.g., the width of the patch on the graph occupied by the point cloud to which it belongs); sizeV0 is the height of the patch to be encoded (e.g., the height of the patch on the graph occupied by the point cloud to which it belongs).

In a possible embodiment, if the to-be-encoded patch is a patch in the first frame point cloud in the subgroup, the to-be-encoded patch is a global matching patch, and the to-be-encoded patch has a patch matching with the global matching patch, the edge information of the to-be-encoded patch is encoded into the code stream, and the matching patch index of the to-be-encoded patch is a non-default value to indicate a reference patch matching with the to-be-encoded patch; the global matching patch mark of the to-be-coded patch is a first value to indicate that the to-be-coded patch is a global matching patch in the subgroup;

if the patch to be coded is a patch in the point cloud of the non-first frame in the subgroup, encoding the side information of the patch to be coded into a code stream, wherein the encoding comprises the following steps: if the to-be-encoded patch is a patch in the non-first frame point cloud in the subgroup, the to-be-encoded patch is a global matching patch and has a patch matched with the global matching patch, encoding side information of the to-be-encoded patch into the code stream, and the matching patch index is a non-default value to indicate a reference patch matched with the to-be-encoded patch. E.g., patch [ k ]. matchedpatcIndex does not equal-1, -1 is a default value, indicating the reference patch of the patch to be encoded.

Considering that the global matching patches in the subgroup have the same two-dimensional information, normal coordinate axis information, rotation transformation information and global matching patch identification, for the patches in the first frame point cloud in the subgroup (for example, patch a, which is the global matching patch, and patch a has the matching patch B), the two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index of patch a are encoded into the code stream, and for the patches in the non-first frame (for example, patch B), the three-dimensional information and matching patch index of patch B are encoded into the code stream. At a decoding end, analyzing the two-dimensional information, the three-dimensional information, the rotation transformation information, the global matching patch identification and the matching patch index of the patch A from the code stream, analyzing the three-dimensional information and the matching patch index of the patch B from the code stream, and determining the global matching patch identification of the patch B according to the global matching patch identification of the patch A if the patch A is determined to be the reference patch of the patch B according to the matching patch index of the patch B. If the global matching patch of the patch B is identified as the first value (e.g., l or tune), the two-dimensional information, the normal coordinate axis information, and the rotation transformation information of the patch B are respectively determined according to the two-dimensional information, the normal coordinate axis information, and the rotation transformation information of the reference patch of the patch B (i.e., the patch a), so that the aforementioned information is prevented from being coded in the code stream, thereby reducing the bit overhead of the code stream and improving the coding efficiency. Note that the normal coordinate information of the patch a does not need to be encoded in the code stream either, but the normal coordinate information of the patch a is determined on the decoding side from the normal coordinate information of the reference patch indicated by the matching patch index of the patch a.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including:

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch and does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises normal coordinate axis information, the side information of the to-be-coded patch is coded into the code stream, and the matching patch index is a default value to indicate that the to-be-coded patch does not have a patch matched with the matching patch. E.g., patch [ k ]. matchedPatchIndex equals-1, -1 is a default value indicating that no patch to be encoded matches it. The global matching patch identifier of the to-be-encoded patch is a first value to indicate that the to-be-encoded patch is the global matching patch in the subgroup.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-coded patch is the patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch, and the to-be-coded patch has a reference patch matched with the to-be-coded patch, encoding side information of the to-be-coded patch into a code stream; the matching patch index of a patch to be encoded is a non-default value to indicate a reference patch that matches the patch to be encoded. E.g., patch [ k ]. matchedpatcindex is not equal to-1 (e.g., -1 is a default value), indicating the reference patch of the patch to be encoded. The global match patch flag of the to-be-encoded patch is a second value to indicate that the to-be-encoded patch is not a global match patch within the subgroup.

If the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, including: if the to-be-encoded patch is a patch in the non-first frame point cloud in the subgroup, the to-be-encoded patch is a non-global matching patch and has a patch matched with the to-be-encoded patch, the side information of the to-be-encoded patch also comprises two-dimensional information, and the side information of the to-be-encoded patch is encoded into the code stream; the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch matching the patch to be encoded. E.g., patch [ k ]. matchedPatchIndex is not equal to-1 (e.g., -1 is a default value), indicating a reference patch to encode the patch.

Considering that the mutually matched patches in the subgroup have the same normal coordinate axis information and rotation transformation information, for a patch in the first frame point cloud in the subgroup (for example, the patch a and the patch a are non-global matching patches and the patch a has a patch B matching therewith), in this embodiment, the two-dimensional information, the three-dimensional information, the rotation transformation information, the global matching patch identifier, and the matching patch index of the patch a are encoded into the code stream, and for a patch in the non-first frame (for example, the patch B), the two-dimensional information, the three-dimensional information, and the matching patch index of the patch B are encoded into the code stream. At a decoding end, two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index of the patch A are analyzed from a code stream, two-dimensional information, three-dimensional information and a matching patch index of the patch B are analyzed from the code stream, once the patch A is determined as a reference patch of the patch B according to the matching patch index of the patch B, and normal coordinate axis information and rotation transformation information of the patch B are respectively determined according to the normal coordinate axis information and the rotation transformation information of the reference patch of the patch B, the information is prevented from being coded in the code stream, so that the bit cost of the code stream can be reduced, and the coding efficiency is improved. It should be noted that, the normal coordinate axis information of the patch a does not need to be encoded into the code stream, but on the decoding side, the normal coordinate axis information of the patch a is determined according to the normal coordinate axis information of the reference patch indicated by the matching patch index of the patch a.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and the to-be-coded patch does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further includes normal coordinate axis information (for example, normaaxis, namely, a normal axis index of the to-be-coded patch), and the side information of the to-be-coded patch is coded into a code stream; the matching patch index of the to-be-encoded patch is a default value to indicate that the to-be-encoded patch does not have a patch that matches it. E.g., patch [ k ]. matchedpatcIndex equals-1, -1 is a default value, indicating that the patch to be encoded does not have a matching patch. The global match patch flag of the to-be-encoded patch is identified as a second value to indicate that the to-be-encoded patch is not a global match patch within the subgroup.

If the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, including: if the to-be-encoded patch is a patch in the non-leading frame point cloud in the subgroup, and the to-be-encoded patch is a non-global matching patch and does not have a patch matching therewith, the side information of the to-be-encoded patch further includes two-dimensional information, normal coordinate axis information (e.g., normaaxis, i.e., a normal axis index of the to-be-encoded patch) and rotation transformation information, and the side information of the to-be-encoded patch is encoded into the code stream, where the matching patch index of the to-be-encoded patch is a default value to indicate that the to-be-encoded patch does not have a patch matching therewith. E.g., patch [ k ]. matchedpatcIndex equals-1, -1 is a default value, indicating that the patch to be encoded does not have a matching patch.

As can be seen from the above, as for the patches (e.g., patch a) in the first frame point cloud, the purpose of encoding the global matching patch ids of the patches into the code stream is that the patches having a matching relationship have the same global matching patch ids. If the patch a is a reference patch of another patch (e.g., the patch B in the non-leading frame), even if the global matching patch id of the patch B is not parsed from the code stream at the decoding end, the global matching patch id of the patch B can be determined according to the global matching patch id of the patch a. If patch B does not have a reference patch that matches it, then patch B may be determined to be a non-global match patch, and its global match patch identification may be directly determined to be a second value (e.g., 0 or false). Therefore, for the global matching patch identification, the global matching patch identification of the patch in the first frame point cloud is only needed to be coded into the code stream at the coding end, and the global matching patch identification of the non-first frame point cloud is not needed to be coded into the code stream, so that the bit cost of the code stream is reduced, and the coding efficiency is improved.

In a feasible implementation mode, the two-dimensional information and/or the three-dimensional information of the patch to be coded are coded into the code stream in a differential coding mode.

It should be noted that encoding two-dimensional information into a code stream by using a differential encoding method specifically encodes a difference between u0 of a patch to be encoded and u0 of a reference patch thereof, a difference between v0 of the patch to be encoded and v0 of the reference patch thereof, a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch thereof, and a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch thereof into the code stream; similarly, encoding the three-dimensional information into the code stream by using a differential encoding method specifically includes encoding a difference between u0 of the patch to be encoded and u0 of the reference patch, a difference between v0 of the patch to be encoded and v0 of the reference patch, a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch, and a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch into the code stream. The differential coding mode can further achieve the purpose of reducing the bit overhead of the code stream.

The reference patch of the to-be-encoded patch is the patch indicated by the matching patch index of the to-be-encoded patch in the point cloud to which the to-be-encoded patch belongs.

It can be seen that, in the scheme of the embodiment of the present application, for the global matching patch, since partial side information of the global matching patch is the same, it is not necessary to encode partial side information of the global matching patch in the non-first frame point cloud in the subgroup into the code stream. By adopting the embodiment of the application, the matching relation between the sub-groups and the rotation transformation information used for restricting the matching patch is consistent is abandoned, so that the packaging of each sub-group is more flexible, and the sizes of the generated depth map and texture map are smaller. Because partial side information of the global matching patch is coded only once, the purpose of improving coding efficiency is achieved, and bit overhead of the code stream is reduced.

In a second aspect, an embodiment of the present application provides a method for decoding a point cloud, including:

analyzing side information of the patch to be decoded from the code stream; if the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index, normal coordinate axis information of the patch to be decoded is obtained according to the matching patch index; if the side information comprises three-dimensional information and matching patch indexes, obtaining global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch indexes; reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded; and the global matching patch identifier is used for indicating whether the to-be-decoded patch is the global matching patch in the subgroup or not.

It can be seen from the above that, at the decoding end, on the one hand, if two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index of the patch a are analyzed from the code stream, even if normal coordinate axis information of the patch a is not analyzed from the code stream, normal coordinate axis information of the patch a can be derived at the decoding side according to the normal coordinate axis information of the reference patch indicated by the matching patch index of the patch a. On the other hand, if it is assumed that the three-dimensional information and the matching patch index of the patch B are analyzed from the code stream, once it is determined that the patch a is the reference patch of the patch B according to the matching patch index of the patch B, the normal coordinate axis information and the rotation transformation information of the patch B are respectively derived or determined according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch of the patch B (i.e., the patch a). Therefore, even if only partial side information of the patch B is analyzed from the code stream, other side information of the patch B can be derived by combining the partial side information of the patch B analyzed and the side information of the patch A analyzed or derived from the code stream by the decoding end, so that the decoding efficiency is improved on the premise of reducing bit overhead of the code stream to a certain extent.

It should be noted that the global match patch flag of the to-be-decoded patch may be a first value (such as tune or 1) or a second value (such as false or 0). The first value of the global matching patch identifier of the to-be-decoded patch indicates that the to-be-decoded patch is the global matching patch, and the second value of the global matching patch identifier of the to-be-decoded patch indicates that the to-be-decoded patch is not the global matching patch in the subgroup.

The matching patch index of the patch to be decoded is the index of the patch that matches the patch to be decoded. The patch matched with the to-be-decoded patch is a patch in a reference frame of the point cloud to which the to-be-decoded patch belongs. The matching patch index of the patch to be decoded may be a non-default value or a default value. The matching patch index of the to-be-decoded patch is a non-default value to indicate a reference patch of the to-be-decoded patch, i.e., the to-be-decoded patch is a matching patch, and the matching patch index of the to-be-decoded patch is a default value to indicate that the to-be-decoded patch does not have a matching patch with the matching patch, i.e., the to-be-decoded patch is a non-matching patch.

The patch in the reference frame indicated by the matching patch index of the patch to be decoded is the reference patch. In other words, the reference patch is the patch indicated by the matching patch index of the patch to be decoded in the reference frame of the point cloud to which the patch to be decoded belongs.

It should be noted that specific descriptions of the two-dimensional information, the three-dimensional information, and the rotation transformation information of the patch to be decoded can be referred to the relevant descriptions of the two-dimensional information, the three-dimensional information, and the rotation transformation information of the patch to be encoded in the first aspect, and are not described herein.

In a possible embodiment, the obtaining, according to the matching patch index, normal coordinate axis information of the patch to be decoded if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and the matching patch index includes:

if the matching patch index is a non-default value, determining normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index;

if the side information includes three-dimensional information and a matching patch index, then global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch are obtained according to the matching patch index, and the method includes the following steps: if the matching patch index is a non-default value, determining a global matching patch identifier of a to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is a first value, respectively determining two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch;

the match patch index is a non-default value to indicate a reference patch of the to-be-decoded patch, and the global match patch flag is a first value to indicate the to-be-decoded patch as a global match patch within the subgroup.

From the above, at the decoding end, on one hand, assuming that two-dimensional information, three-dimensional information, normal coordinate axis information, rotation transformation information, global matching patch identification and matching patch index of the patch a are analyzed from the code stream, even if the normal coordinate axis information of the patch a is not analyzed from the code stream, the normal coordinate axis information of the patch a can be derived at the decoding side according to the normal coordinate axis information of the reference patch indicated by the matching patch index of the patch a. On the other hand, assuming that the three-dimensional information and the matching patch index of the patch B are analyzed from the code stream, if the reference patch of the patch B is determined according to the matching patch index of the patch B, the global matching patch identifier of the patch B is determined according to the global matching patch identifier of the patch a, and if the global matching patch identifier of the patch B is the first value, the normal coordinate axis information and the rotation transformation information of the patch B are respectively derived or determined according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch of the patch B (i.e., the patch a). Therefore, even if only partial side information of the patch B is analyzed from the code stream, other side information of the patch B can be obtained by combining the partial side information of the patch B with the side information of the patch A which is analyzed or derived from the code stream by the decoding end, so that the decoding efficiency is improved on the premise of reducing bit overhead of the code stream to a certain extent.

In a possible embodiment, the obtaining, according to the matching patch index, normal coordinate axis information of the patch to be decoded if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and the matching patch index includes: if the matching patch index is a default value, normal coordinate axis information of the patch to be decoded is analyzed from the code stream;

the match patch index is a default value to indicate that the to-be-decoded patch has no matched patch, and the global match patch flag is a first value to indicate that the to-be-decoded patch is a global match patch in the subgroup.

In a possible embodiment, if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, the obtaining, according to the matching patch index, normal coordinate axis information of the patch to be decoded includes: if the matching patch index is a non-default value, determining normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index;

if the side information comprises three-dimensional information and a matching patch index, obtaining global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the matching patch index, and the method comprises the following steps: if the matching patch index is a non-default value, determining a global matching patch identifier of a to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is a second value (for example false or 0), analyzing two-dimensional information of the to-be-decoded patch from the code stream, and respectively determining normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the normal coordinate axis information and the rotation transformation information of the reference patch;

wherein the matching patch index of the to-be-decoded patch is a non-default value to indicate a reference patch of the to-be-decoded patch; the global match patch flag of the to-be-decoded patch is a second value to indicate that the to-be-decoded patch is not a global match patch within the subgroup.

It can be seen from the above that, at the decoding end, on the one hand, if two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index of the patch a are analyzed from the code stream, even if normal coordinate axis information of the patch a is not analyzed from the code stream, normal coordinate axis information of the patch a can be derived at the decoding side according to the normal coordinate axis information of the reference patch indicated by the matching patch index of the patch a. On the other hand, assuming that the three-dimensional information and the matching patch index of the patch B are analyzed from the code stream, if the reference patch of the patch B is determined according to the matching patch index of the patch B, the global matching patch of the patch B is determined according to the global matching patch identifier of the patch a, and if the global matching patch identifier of the patch B is a second value, the two-dimensional information of the patch B is also analyzed from the code stream, and the normal coordinate axis information and the rotation transformation information of the patch B are respectively derived or determined according to the normal coordinate axis information and the rotation transformation information of the reference patch of the patch B (i.e., the patch a). Therefore, even if only partial side information of the patch B is analyzed from the code stream, other side information of the patch B can be obtained by combining the partial side information of the patch B with the side information of the patch A which is analyzed or derived from the code stream by the decoding end, so that the decoding efficiency is improved on the premise of reducing bit overhead of the code stream to a certain extent.

In a possible embodiment, the obtaining, according to the matching patch index, normal coordinate axis information of the patch to be decoded if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and the matching patch index includes: if the matched patch index is a default value, analyzing the information of the normal coordinate axis of the patch to be decoded from the code stream;

if the side information includes three-dimensional information and a matching patch index, then global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch are obtained according to the matching patch index, and the method includes the following steps: if the matching patch index is a default value, analyzing two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch from the code stream, and determining that the global matching patch identifier of the to-be-decoded patch is a second value;

the matching patch index of the to-be-decoded patch is a default value to indicate that the to-be-decoded patch has no patch matched with the to-be-decoded patch, and the global matching patch identifier of the to-be-decoded patch is a second value to indicate that the to-be-decoded patch is not the global matching patch in the subgroup.

As can be seen from the above, at the decoding end, the technical effect of determining the global matching patch identifier of the current patch (such as patch a) as the second value (e.g. 0 or false) is: if the reference patch of the patch B is the patch A, determining the global matching patch identifier of the patch B according to the global matching patch identifier of the patch A, thereby determining that the patch B is the non-global matching patch, further analyzing the two-dimensional information of the patch B from the code stream, and respectively deducing or determining the normal coordinate axis information and the rotation transformation information of the patch B according to the normal coordinate axis information and the rotation transformation information of the reference patch (namely, the patch A) of the patch B. Therefore, even if only partial side information of the patch B is analyzed from the code stream, other side information of the patch B can be obtained by combining the partial side information of the patch B with the side information of the patch A which is analyzed or derived from the code stream by the decoding end, so that the decoding efficiency is improved on the premise of reducing bit overhead of the code stream to a certain extent.

In one possible embodiment, a differential decoding mode is adopted to analyze the two-dimensional information and/or the three-dimensional information of the patch to be decoded from the code stream.

It should be noted that, in the differential decoding, the two-dimensional information of the to-be-decoded patch is analyzed from the codestream, specifically, the difference between u0 of the to-be-decoded patch and u0 of the reference patch thereof, the difference between v0 of the to-be-decoded patch and v0 of the reference patch thereof, the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch thereof, and the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch thereof are analyzed from the codestream; then, u0 of the to-be-decoded patch is obtained according to the difference between u0 of the to-be-decoded patch and u0 of the reference patch and u0 of the reference patch, v0 of the to-be-decoded patch is obtained according to the difference between v0 of the to-be-decoded patch and v0 of the reference patch and v0 of the reference patch, sizeU0 of the to-be-decoded patch is obtained according to the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch and sizeU0 of the reference patch, and sizeU0 of the to-be-decoded patch is obtained according to the difference between sizeU0 of the to-decoded patch and sizeU0 of the reference patch and sizeU0 of the reference patch. Similarly, the analyzing the three-dimensional information of the to-be-decoded patch from the code stream by adopting the differential decoding method specifically includes analyzing a difference value between u1 of the to-be-decoded patch and u1 of a reference patch thereof from the code stream, a difference value between v1 of the to-be-decoded patch and v1 of the reference patch thereof, and a difference value between d1 of the to-be-decoded patch and d1 of the reference patch thereof; then, u1 of the patch to be decoded is obtained according to the difference between u1 of the patch to be decoded and u1 of the reference patch and u1 of the reference patch, v1 of the patch to be decoded is obtained according to the difference between v1 of the patch to be decoded and v1 of the reference patch and v1 of the reference patch, and d1 of the patch to be decoded is obtained according to the difference between d1 of the patch to be decoded and d1 of the reference patch and d1 of the reference patch.

It can be seen that, in the scheme of the embodiment of the application, the side information of the global matching patch in the point cloud of the sub-group head frame is analyzed from the code stream; because the partial side information of the global matching patch in the non-head frame point is the same, the partial side information of the global matching patch in the non-head frame point can be obtained by directly deducing according to the side information of the global matching patch in the head frame point cloud, and the decoding efficiency is improved.

In a third aspect, an embodiment of the present application provides another method for encoding a point cloud, including:

acquiring side information of a patch to be coded;

encoding indication information into a code stream, wherein the indication information is used for indicating whether side information of a patch to be encoded is encoded according to a target encoding method; the object encoding method comprises part or all of the point cloud encoding method as described in the first aspect.

In a fourth aspect, an embodiment of the present application provides another method for decoding point clouds, including:

analyzing the code stream to obtain indication information, wherein the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method comprises part or all of the point cloud encoding method as described in the second aspect; and reconstructing the patch to be decoded according to the side information of the patch to be decoded.

In a fifth aspect, an embodiment of the present application further provides an apparatus for encoding a point cloud, including:

the patch information acquisition module is used for acquiring the side information of the patch to be coded;

the auxiliary information encoding module is used for encoding the side information of the patch to be encoded into the code stream if the patch to be encoded is the patch in the first frame point cloud in the subgroup; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index; the global matching patch identifier is used for indicating whether the to-be-coded patch is a global matching patch in the subgroup or not; if the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding side information of the patch to be coded into a code stream, wherein the side information comprises three-dimensional information and a matching patch index.

In a possible embodiment, in the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the first frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch, and the to-be-coded patch has a patch matched with the global matching patch, encoding the side information of the to-be-coded patch into a code stream; the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch that matches the patch to be encoded; the global matching patch identifier of the to-be-coded patch is a first value to indicate that the to-be-coded patch is a global matching patch in the subgroup;

in the aspect of encoding side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in a non-first-frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to: if the to-be-coded patch is a patch in the non-first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch, and the to-be-coded patch has a patch matched with the global matching patch, encoding side information of the to-be-coded patch into a code stream; the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch that matches the patch to be encoded.

In one possible embodiment, in the aspect that if the patch to be encoded is a patch in the first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the code stream, the auxiliary information encoding module is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a globally matched patch and the to-be-coded patch does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises normal coordinate axis information, and the side information of the to-be-coded patch is coded into the code stream; wherein, the matching patch index of the patch to be coded is a default value to indicate that the patch to be coded does not have a patch matched with the matching patch; the global match patch flag of the to-be-encoded patch is a first value to indicate that the to-be-encoded patch is a global match patch within the subgroup.

In a possible embodiment, in the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the first frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to: if the to-be-encoded patch is the patch in the first frame point cloud in the subgroup, the to-be-encoded patch is a non-global matching patch and has a patch matched with the to-be-encoded patch, encoding side information of the to-be-encoded patch into a code stream; the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch matching the patch to be encoded; the global matching patch mark of the to-be-coded patch is a second value to indicate that the to-be-coded patch is not the global matching patch in the subgroup;

in the aspect of encoding side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in a non-first-frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and has a patch matched with the to-be-coded patch, the side information of the to-be-coded patch also comprises two-dimensional information, and the side information of the to-be-coded patch is coded into the code stream; the matching patch index of the to-be-encoded patch is a non-default value to indicate a reference patch that matches the to-be-encoded patch.

In a possible embodiment, in the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the first frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and the to-be-coded patch does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch also comprises normal coordinate axis information, and the side information of the to-be-coded patch is coded into a code stream; the matching patch index of the to-be-coded patch is a default value to indicate that the to-be-coded patch has no patch matched with the to-be-coded patch; the global matching patch mark of the to-be-coded patch is a second value to indicate that the to-be-coded patch is not the global matching patch in the subgroup;

in the aspect of encoding side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in a non-first-frame point cloud in the subgroup, the auxiliary information encoding module is specifically configured to:

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, normal coordinate axis information and rotation transformation information, and the side information of the to-be-coded patch is coded into the code stream; the matching patch index of the to-be-encoded patch is a default value to indicate that the to-be-encoded patch does not have a patch matching therewith.

In one possible embodiment, the two-dimensional information and/or the three-dimensional information of the patch to be coded are coded into the code stream in a differential coding mode; the two-dimensional information comprises u0, v0, sizeU0, sizeV0 and u0, wherein the abscissa value of the patch to be coded at the left upper corner of the map occupied by the point cloud to which the patch belongs, and v0 is the abscissa value of the patch to be coded at the left upper corner of the map occupied by the point cloud to which the patch belongs; the sizeU0 is the width of the patch to be encoded on the point cloud occupancy map to which the patch belongs; the sizeV0 is the height of the patch to be encoded on the point cloud occupancy map to which the patch belongs;

the three-dimensional information includes u1, d1, and v1, u1 (also referred to as patch _3d _ shift _ tangent _ axis) is the offset of the point in the patch along the tangent axis, v1 (also referred to as patch _3d _ shift _ bitmap _ axis) is the offset of the point in the patch along the bisector axis, and d1 (also referred to as patch _3d _ shift _ normal _ axis) is the offset of the point in the patch along the normal axis.

In a sixth aspect, an embodiment of the present application further provides a device for decoding a point cloud, including:

the auxiliary information decoding module is used for analyzing the side information of the patch to be decoded from the code stream;

the auxiliary information decoding module is further used for obtaining normal coordinate axis information of the patch to be decoded according to the global matching patch and the matching patch index of the patch to be decoded if the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index; if the side information comprises three-dimensional information and matching patch indexes, determining global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch indexes of the patch to be decoded;

the reconstruction module is used for reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded; the global matching patch identifier is used for indicating whether the to-be-decoded patch is the global matching patch in the subgroup.

In a possible embodiment, in the aspect that if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, the normal coordinate axis information of the patch to be decoded is determined according to the matching patch index, the auxiliary information decoding module is specifically configured to:

if the matching patch index of the to-be-decoded patch is determined to be a non-default value, determining normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index;

if the side information includes three-dimensional information and a matching patch index, determining the global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch index of the patch to be decoded, wherein the auxiliary information decoding module is specifically used for: if the matching patch index of the to-be-decoded patch is determined to be a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is a first value, respectively determining two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch; the global matching patch identifier of the to-be-decoded patch is a first value to indicate that the to-be-decoded patch is a global matching patch in the subgroup; the reference patch is a patch which is indicated by the matching patch index in the reference frame of the point cloud to which the patch to be decoded belongs.

In a possible embodiment, if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, in an aspect that normal coordinate axis information of a to-be-decoded patch is determined according to the to-be-matched patch index, the auxiliary information decoding module is specifically configured to:

if the matching patch index of the to-be-decoded patch is determined to be a default value, normal coordinate axis information of the to-be-decoded patch is analyzed from the code stream; the matching patch index of the to-be-decoded patch is a default value to indicate that the to-be-decoded patch does not have a patch matching therewith.

if the side information comprises three-dimensional information and matching patch indexes, determining global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the matching patch indexes of the to-be-decoded patch, wherein the auxiliary information decoding module is specifically used for:

if the matching patch index of the to-be-decoded patch is determined to be a non-default value; determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is determined to be a second value; then, the two-dimensional information of the patch to be decoded is analyzed from the code stream; determining normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the normal coordinate axis information and the rotation transformation information of the reference patch indicated by the matching patch index; the matching patch index of the to-be-decoded patch is a non-default value to indicate a reference patch of the to-be-decoded patch, and the global matching patch identifier of the to-be-decoded patch is a second value to indicate that the to-be-decoded patch is not the global matching patch in the subgroup; the reference patch is a patch indicated by a matching patch index in the reference frame of the point cloud to which the patch to be decoded belongs.

if the matching patch index of the to-be-decoded patch is determined to be a default value, normal coordinate axis information of the to-be-decoded patch is analyzed from the code stream;

if the side information includes three-dimensional information and a matching patch index, determining the global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch index of the patch to be decoded, wherein the auxiliary information decoding module is specifically used for: if the matching patch index of the to-be-decoded patch is determined to be a default value, analyzing two-dimensional information, normal coordinate axis information and rotation transformation information from the code stream, and determining a global matching patch identifier of the to-be-decoded patch to be a second value;

In one possible embodiment, encoding the two-dimensional information and/or the three-dimensional information of the patch to be decoded into a code stream in a differential decoding mode; the two-dimensional information comprises u0, v0, sizeU0, sizeV0 and u0, wherein the u0, v0, sizeU0, sizeV0 and u0 are abscissa values of the to-be-decoded patch in the upper left corner of the to-be-decoded patch in the graph, and v0 is an abscissa value of the to-be-decoded patch in the upper left corner of the to-be-decoded patch in the graph; sizeU0 is the width of the patch to be decoded on the point cloud occupancy map to which the patch belongs; the sizeV0 is the height of the patch to be decoded on the point cloud occupancy map to which the patch belongs;

In a seventh aspect, an embodiment of the present application provides another apparatus for encoding a point cloud, including:

the patch information acquisition module is used for acquiring the side information of the to-be-coded patch;

the auxiliary information coding module is used for coding indication information into a code stream, wherein the indication information is used for indicating whether side information of the patch to be coded is coded according to a target coding method; the object encoding method comprises part or all of the point cloud encoding method of the first aspect of the claims.

In an eighth aspect, an embodiment of the present application provides another apparatus for decoding a point cloud, including:

the auxiliary information decoding module is used for analyzing the code stream to obtain indication information, and the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method comprises part or all of the point cloud decoding method of the second aspect; the described

And the reconstruction module is used for reconstructing the patch to be decoded according to the side information of the patch to be decoded.

In a ninth aspect, an apparatus for encoding point cloud data is provided, which may include: a memory and an encoder. The memory is used for storing point cloud data. The encoder is configured to perform the point cloud encoding method provided by the first aspect or any one of the possible designs of the first aspect.

In a tenth aspect, an apparatus for decoding point cloud data is provided, which may include: a memory and a decoder. The memory is used for storing point cloud data in a code stream form. The decoder is configured to perform the point cloud encoding method provided by the second aspect or any one of the possible designs of the second aspect.

In an eleventh aspect, there is provided an encoding apparatus comprising: a memory and a processor; wherein the memory is used for storing program codes; the processor is configured to invoke the program code to execute the point cloud encoding method provided by the first aspect.

In a twelfth aspect, there is provided a decoding apparatus comprising: a memory and a processor; wherein the memory is used for storing program codes; the processor is configured to invoke the program code to execute the point cloud decoding method provided by the second aspect.

The present application also provides a computer-readable storage medium comprising program code which, when run on a computer, causes the computer to perform any one of the point cloud decoding methods as provided by the first aspect and its possible designs, or the second aspect and its possible designs, as described above.

The present application also provides a computer-readable storage medium comprising program code which, when run on a computer, causes the computer to execute the point cloud encoding method provided by the third aspect described above.

The present application also provides a computer-readable storage medium comprising program code which, when run on a computer, causes the computer to execute the point cloud decoding method provided by the fourth aspect described above.

The present application also provides a computer program product which, when run on a computer, causes the computer to perform some or all of the steps of the point cloud encoding method provided by the first aspect or any of the possible designs of the first aspect.

The present application also provides a computer program product which, when run on a computer, causes the computer to perform some or all of the steps of the point cloud decoding method provided by the second aspect or any one of the possible designs of the second aspect.

It should be understood that beneficial effects of any one of the codecs, the apparatuses, the codec apparatuses, and the computer readable storage medium provided above may correspond to the beneficial effects of the method embodiments provided with reference to the above corresponding aspects, and are not described again.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic block diagram of a point cloud coding system that may be used for one example of an embodiment of the present application;

FIG. 2 is a schematic block diagram of an encoder that may be used in one example of an embodiment of the present application;

FIG. 3 is a schematic diagram of a point cloud, a patch of the point cloud, and an occupancy map of the point cloud that may be used in embodiments of the present application;

FIG. 4 is a schematic block diagram of a decoder that may be used for one example of an embodiment of the present application;

FIG. 5 is a schematic diagram of a subgroup in an embodiment of the present application;

fig. 6 is a schematic flowchart of a point cloud encoding method according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart of a point cloud decoding method according to an embodiment of the present disclosure;

fig. 8 is a schematic diagram of a specific point cloud encoding/decoding provided in the embodiment of the present application;

FIG. 9 is a schematic structural diagram of an apparatus for encoding a point cloud according to the present application

Fig. 10 is a schematic structural diagram of a point cloud decoding apparatus according to the present application;

fig. 11 is a schematic structural diagram of an apparatus for encoding a point cloud according to the present application;

fig. 12 is a schematic structural diagram of a point cloud decoding apparatus according to the present application;

fig. 13 is a schematic block diagram of an implementation manner of a decoding apparatus according to an embodiment of the present application.

Detailed Description

The term "at least one" in the embodiments of the present application includes one or more. "plurality" means two (species) or more than two (species). For example, at least one of A, B and C, comprising: a alone, B alone, a and B in combination, a and C in combination, B and C in combination, and A, B and C in combination. The term "and/or" in the embodiment of the present application is only one kind of association relation describing an association object, and indicates that three kinds of relations may exist, for example, a and/or B, and may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The term "/" in the embodiments of the present application generally means that the former and latter associated objects are in an "or" relationship. In the formula, the character "/" represents a division operation, such as A/B represents A divided by B. The terms "first", "second", and the like in the embodiments of the present application are used for distinguishing different objects, and do not limit the order of the different objects.

Fig. 1 is a schematic block diagram of a point cloud coding system 1 that may be used for one example of an embodiment of the present application. The term "point cloud coding" or "coding" may generally refer to point cloud encoding or point cloud decoding. The encoder 100 of the point cloud decoding system 1 may encode the point cloud to be encoded according to any one of the point cloud encoding methods proposed in the present application. The decoder 200 of the point cloud decoding system 1 can decode the point cloud to be decoded according to the point cloud decoding method corresponding to the point cloud encoding method used by the encoder.

As shown in fig. 1, the point cloud decoding system 1 includes a source device 10 and a destination device 20. Source device 10 generates encoded point cloud data. Accordingly, the source device 10 may be referred to as a point cloud encoding device. Destination device 20 may decode the encoded point cloud data generated by source device 10. Accordingly, the destination device 20 may be referred to as a point cloud decoding device. Various implementations of source device 10, destination device 20, or both may include one or more processors and memory coupled to the one or more processors. The memory can include, but is not limited to, Random Access Memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein.

Source device 10 and destination device 20 may comprise a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

Destination device 20 may receive the encoded point cloud data from source device 10 via link 30. Link 30 may comprise one or more media or devices capable of moving the encoded point cloud data from source device 10 to destination device 20. In one example, link 30 may comprise one or more communication media that enable source device 10 to send encoded point cloud data directly to destination device 20 in real-time. In this example, source device 10 may modulate the encoded point cloud data according to a communication standard, such as a wireless communication protocol, and may send the modulated point cloud data to destination device 20. The one or more communication media may include wireless and/or wired communication media such as a Radio Frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the internet). The one or more communication media may include a router, switch, base station, or other apparatus that facilitates communication from source device 10 to destination device 20.

In another example, encoded data may be output from output interface 140 to storage device 40. Similarly, encoded point cloud data may be accessed from storage device 40 through input interface 240. Storage device 40 may include any of a variety of distributed or locally accessed data storage media such as a hard drive, blu-ray discs, Digital Versatile Discs (DVDs), compact disc read-only memories (CD-ROMs), flash memories, volatile or non-volatile memories, or any other suitable digital storage medium for storing encoded point cloud data.

In another example, storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded point cloud data generated by source device 10. Destination device 20 may access the stored point cloud data from storage device 40 via streaming or download. The file server may be any type of server capable of storing the encoded point cloud data and sending the encoded point cloud data to the destination device 20. Example file servers include network servers (e.g., for websites), File Transfer Protocol (FTP) servers, Network Attached Storage (NAS) devices, or local disk drives. Destination device 20 may access the encoded point cloud data through any standard data connection, including an internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both suitable for accessing encoded point cloud data stored on a file server. The transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.

The point cloud coding system 1 illustrated in fig. 1 is merely an example, and the techniques of this application may be applicable to point cloud coding (e.g., point cloud encoding or point cloud decoding) devices that do not necessarily include any data communication between the point cloud encoding device and the point cloud decoding device. In other examples, the data is retrieved from local storage, streamed over a network, and so forth. The point cloud encoding device may encode and store data to a memory, and/or the point cloud decoding device may retrieve and decode data from a memory. In many examples, encoding and decoding are performed by devices that do not communicate with each other, but rather only encode data to and/or retrieve data from memory and decode data.

In the example of fig. 1, source device 10 includes a data source 110, an encoder 100, and an output interface 140. In some examples, output interface 140 can include a regulator/demodulator (modem) and/or a transmitter (otherwise referred to as a transmitter). The data source 110 may include a point cloud capture device (e.g., a camera), a point cloud archive containing previously captured point cloud data, a point cloud feed interface to receive point cloud data from a point cloud content provider, and/or a computer graphics system for generating point cloud data, or a combination of these sources of point cloud data.

The encoder 100 may encode point cloud data from a data source 110. In some examples, source device 10 sends the encoded point cloud data directly to destination device 20 via output interface 140. In other examples, the encoded point cloud data may also be stored onto storage device 40 for later access by destination device 20 for decoding and/or playback.

In the example of fig. 1, destination device 20 includes input interface 240, decoder 200, and display device 120. In some examples, input interface 240 includes a receiver and/or a modem. The input interface 240 may receive the encoded point cloud data via the link 30 and/or from the storage device 40. Display device 120 may be integrated with destination device 20 or may be external to destination device 20. In general, the display device 120 displays the decoded point cloud data. The display device 120 may include various display devices, such as a Liquid Crystal Display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

Although not shown in fig. 1, in some aspects, encoder 100 and decoder 200 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to handle encoding of both audio and video in a common data stream or separate data streams. In some examples, the MUX-DEMUX unit may conform to the ITU h.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

Encoder 100 and decoder 200 may each be implemented as any of a variety of circuits, such as: one or more microprocessors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combinations thereof. If the present application is implemented in part in software, a device may store instructions for the software in a suitable non-volatile computer-readable storage medium and may execute the instructions in hardware using one or more processors to implement the techniques of the present application. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (codec) in the respective device.

This application may refer generally to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The terms "signaling" or "sending" may generally refer to the transfer of syntax elements and/or other data used to decode the compressed point cloud data. This transfer may occur in real time or near real time. Alternatively, such communication may occur over a period of time, such as may occur when syntax elements are stored to a computer-readable storage medium in an encoded bitstream at the time of encoding, which the decoding device may then retrieve at any time after the syntax elements are stored to such medium.

As shown in fig. 2, is a schematic block diagram of an encoder 100 that may be used in one example of an embodiment of the present application. Fig. 2 is an example of an mpeg (moving picture expert group) Point Cloud Compression (PCC) encoding framework. In the example of fig. 2, the encoder 100 may include a patch information generating module 101, a packing module 102, a depth map generating module 103, a texture map generating module 104, a first padding module 105, an image or video based encoding module 106, an occupancy map encoding module 107, an auxiliary information encoding module 108, and a multiplexing module 109, etc. In addition, the encoder 100 may further include a point cloud filtering module 110, a second filling module 111, a point cloud reconstruction module 112, and the like. Wherein:

the patch information generating module 101 is configured to generate a plurality of patches by dividing a frame point cloud by a certain method, and obtain information related to the generated patches. The patch refers to a set of partial points in a frame point cloud, and usually one connected region corresponds to one patch. The information related to patch may include, but is not limited to, at least one of the following: the number of patches into which the point cloud is divided, the position information of each patch in the three-dimensional space, the index of the normal coordinate axis of each patch, the depth map generated by projecting each patch from the three-dimensional space to the two-dimensional space, the size of the depth map (for example, the width and height of the depth map) of each patch, the occupancy map generated by projecting each patch from the three-dimensional space to the two-dimensional space, and the like. In one example, the parts of the related information, such as the number of patches into which the point cloud is divided, the index of the normal coordinate axis of each patch, the depth map size of each patch, the position information of each patch in the point cloud, the size information of the occupancy map of each patch, and the like, may be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (i.e., compression encoding). In another example, parts of the related information, such as a part of two-dimensional information (e.g., sizeU0 and sizeuv 0) of a patch generated by point cloud segmentation, three-dimensional information (e.g., u1, d1 and v1), and even a global matching patch identification (e.g., patch [ k ]. globalpatchglag) of the patch in the first frame point cloud in the subgroup, may be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (i.e., compression encoding). In yet another example, portions of the related information, such as three-dimensional information of a patch generated by the point cloud being segmented (e.g., u1, d1, and v1) may be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (i.e., compression encoding). It should be noted that the auxiliary information encoding module 108 may employ different encoding methods (e.g., fixed-length encoding or differential encoding) to encode or write the side information into the code stream, which will be described in detail later in this application.

The occupancy map of each patch may be sent to the packing module 102 for packing, specifically, the patches of the point cloud are arranged in a specific order, for example, in a descending order (or ascending order) of the width/height of the occupancy map of each patch; then, according to the sequence of each arranged patch, the occupancy map of the patch is inserted into the available area of the point cloud occupancy map in sequence to obtain the occupancy map of the point cloud. On the other hand, specific position information of each patch in the point cloud occupancy map, a depth map of each patch, and the like may be sent to the depth map generation module 103. In some implementations, the packing module 102 may pack the point clouds included in the plurality of subgroups respectively by using the divided subgroups as units, so as to obtain an occupancy map of each point cloud in the point cloud group to be encoded. The packing algorithm used for packing the point clouds in a subgroup is generally the same. The point cloud in different subgroups can be packed by the same or different packing algorithms, which is not limited in the embodiments of the present application.

After the packing module 102 obtains the occupancy map of the point cloud, on one hand, the occupancy map of the point cloud may be filled by the second filling module 111 and then sent to the occupancy map encoding module 107 for encoding. On the other hand, the occupancy map of the point cloud may be used to guide the depth map generation module 103 to generate a depth map of the point cloud and guide the texture map generation module 104 to generate a texture map of the point cloud. On the other hand, the packing module 102 may further obtain a matching patch index (e.g., patch [ k ]. matchedPatchIndex), rotation transformation information (e.g., patch [ k ]. orintiationIndex) and another part of two-dimensional information (e.g., u0 and v0) of the patch, and send the matching patch index (e.g., patch [ k ]. matchedPatchIndex), rotation transformation information (e.g., patch [ k ]. orintiationIndex) and another part of two-dimensional information (e.g., u0 and v0) of the patch to the side information encoding module 108 for encoding.

Fig. 3 is a schematic diagram of a point cloud, a patch of the point cloud, and an occupation map of the point cloud, which are applicable to the embodiment of the present application. Fig. 3 (a) is a schematic diagram of a frame of point cloud, fig. 3 (b) is a schematic diagram of a patch of the point cloud obtained based on fig. 3 (a), and fig. 3 (c) is a schematic diagram of an occupancy map of the point cloud obtained by packing an occupancy map of each patch mapped onto a two-dimensional plane as shown in fig. 3 (b).

The depth map generating module 103 is configured to generate a depth map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud, and the depth information, and send the generated depth map to the first filling module 105, so as to fill blank pixels in the depth map, thereby obtaining a filled depth map.

The texture map generating module 104 is configured to generate a texture map of the point cloud according to the occupancy map of the point cloud, the occupancy maps of the respective patches of the point cloud, and the texture information, and send the generated texture map to the first filling module 105 to fill the blank pixels in the texture map, so as to obtain a filled texture map.

The padded depth map and the padded texture map are sent by the first padding module 105 to the image or video based encoding module 106 for image or video based encoding. And (3) subsequent:

in one aspect, the image or video based encoding module 106, the occupancy map encoding module 107, and the auxiliary information encoding module 108 send the obtained encoding results (i.e., the code stream) to the multiplexing module 109 to be combined into a code stream, which can be sent to the output interface 140.

On the other hand, the encoding result (i.e. code stream) obtained by the image or video based encoding module 106 is sent to the point cloud reconstruction module 112 for point cloud reconstruction, so as to obtain a reconstructed point cloud (i.e. reconstructed point cloud geometric information). Specifically, the video decoding is performed on the coded depth map obtained by the image or video-based coding module 106 to obtain a decoded depth map of the point cloud; the reconstructed point cloud geometry information (e.g., patches in the reconstructed point cloud) is obtained using the decoded depth map, the occupancy map of the point cloud, and the auxiliary information for each patch. The geometric information of the point cloud refers to coordinate values of points in the point cloud (e.g., each point in the point cloud) in a three-dimensional space.

Optionally, the point cloud reconstructing module 112 may further send texture information of the point cloud and the reconstructed point cloud geometric information to the coloring module, and the coloring module is configured to color the reconstructed point cloud to obtain the texture information of the reconstructed point cloud.

Optionally, the texture map generating module 104 may further generate a texture map of the point cloud based on information obtained by filtering the reconstructed point cloud geometric information through the point cloud filtering module 110.

It is understood that the encoder 100 shown in fig. 2 is merely an example, and in particular implementations, the encoder 100 may include more or fewer modules than shown in fig. 2. This is not limited in the embodiments of the present application.

As shown in fig. 4, is a schematic block diagram of a decoder 200 that may be used in one example of an embodiment of the present application. In fig. 4, an MPEG PCC decoding framework is illustrated as an example. In the example of fig. 4, the decoder 200 may include a demultiplexing module 201, an image or video based decoding module 202, an occupancy map coding module 203, an auxiliary information decoding module 204, a geometrical information reconstruction module 205 of the point cloud, a point cloud filtering module 206, and a texture information reconstruction module 207 of the point cloud. Wherein:

the demultiplexing module 201 is configured to send the input code stream (i.e., the merged code stream) to a corresponding decoding module. Specifically, a code stream containing the encoded texture map and the encoded depth map is sent to the image or video-based decoding module 202; the code stream containing the encoded occupancy map is sent to the occupancy map decoding module 203, and the code stream containing the encoded auxiliary information is sent to the auxiliary information decoding module 204.

An image or video based decoding module 202 for decoding the received encoded texture map and encoded depth map; and sends the texture map information obtained by decoding to the texture information reconstruction module 207 of the point cloud, and sends the depth map information obtained by decoding to the geometric information reconstruction module 205 of the point cloud. And the occupancy map decoding module 203 is configured to decode the received code stream including the encoded occupancy map, and send the occupancy map information obtained by decoding to the geometrical information reconstruction module 205 of the point cloud. And an auxiliary information decoding module 204, configured to decode the received encoded auxiliary information, and send the decoded information indicating the auxiliary information to a geometrical information reconstruction module 205 of the point cloud. In an example, the side information decoding module 204 decodes the side information, such as two-dimensional information (e.g., u0, v0, sizeU0, and sizeU 0) of the patch to be decoded, three-dimensional information (e.g., u1, d1, and v1), rotation transformation information (e.g., patch [ k ]. orientationIndex), a matching patch index (e.g., patch [ k ]. matchedpatch index), and even a global matching patch identifier (e.g., patch [ k ]. globpaltchflag) of the patch in the first frame point cloud in the subgroup; in another example, the side information decoding module 204 decodes the side information, such as the three-dimensional information of the patch to be decoded (e.g., u1, d1, and v1), and the matching patch index (e.g., patch [ k ]. matchedPatchIndex). It should be noted that the auxiliary information decoding module 204 may use different decoding methods (e.g., fixed length decoding or differential decoding) to parse or decode the side information from the code stream, which will be described in detail later in this application.

A geometric information reconstruction module 205 of the point cloud, configured to reconstruct geometric information of the point cloud (for example, reconstruct a patch in the point cloud) according to the received occupancy map information, depth map information, and auxiliary information. The reconstructed geometrical information of the point cloud is filtered by the point cloud filtering module 206 and then sent to the texture information reconstructing module 207 of the point cloud.

And the point cloud texture information reconstruction module 207 is used for reconstructing the point cloud texture information to obtain reconstructed point cloud.

It is understood that the decoder 200 shown in fig. 4 is merely an example, and in particular implementations, the decoder 200 may include more or fewer modules than those shown in fig. 4. This is not limited in the embodiments of the present application.

In order to facilitate understanding of the technical solutions provided in the embodiments of the present application, the following description will be made of the technologies and terms related to the embodiments of the present application.

In the MPEG point cloud encoding method, an encoder first divides a point cloud to be encoded (i.e., a current frame or a current frame point cloud) into a plurality of patches according to a certain criterion, and the patches have no overlapping area with each other. Then, each patch is projected from the three-dimensional space to a two-dimensional plane, resulting in a two-dimensional image (i.e., the occupancy map of the patch). Next, the occupancy maps of all the slots (or the occupancy maps of the slots with the reduced resolution) are arranged on a two-dimensional image according to a certain rule, so as to obtain the current frame occupancy map. This method of arranging the occupancy map of the patch is called packing. And subsequently, generating a current frame depth map and a current frame texture map according to the packing sequence. The current frame depth map is a two-dimensional image generated by the depth of each patch through projection according to a packing sequence. The texture map of the current frame is a two-dimensional image generated by the texture maps obtained by projection of the patches according to the packing sequence. The current frame occupancy map is a 2-value two-dimensional image for indicating whether each pixel position of the two-dimensional image is occupied by a certain point in the point cloud. In general, to save coding bit overhead, the resolution of the current frame occupancy map is lower than the resolution of the current frame depth map and the current frame texture map.

The cloud group of points to be encoded comprises a plurality of frames of point clouds, which can be continuous or discontinuous in time. For example, the point cloud group to be encoded may be one or more group of frames (GOFs), and a GOF may include 32 frame point clouds, but is not limited thereto.

It should be noted that, the point clouds in a point cloud group are divided into a plurality of sub-groups, and each sub-group includes one or more frames of point clouds. As shown in fig. 5, a point cloud set (comprising 32 frame point clouds) is divided into a plurality of sub-sets. The 0 th sub-group comprises 01 th, 2 th and 3 th frame point clouds in the point cloud group, the 1 st sub-group comprises 4 th and 5 th frame point clouds in the point cloud group, the 2 nd sub-group comprises 6 th, 7 th, 8 th, 9 th and 10 th frame point clouds in the point cloud group, and the 3 rd sub-group comprises 11 th, 12 th, 13 th, 14 th and 15 th frame point clouds in the point cloud group. The point clouds included in the 0 th, 1 st, 2 nd, 3 rd subgroups are given in fig. 5 by way of example only, and the point clouds included in the other subgroups are not specifically described. Any one of the subgroups into which the point cloud to be encoded is divided may contain one or more frames of the point cloud. The number of frames of the point clouds included in different subgroups may be equal or unequal. And a method of dividing the point cloud group to be encoded into a plurality of sub-groups, which is not limited in the present application.

The matching patch in this application refers to a patch that finds a matching patch (also referred to as a reference patch) in a reference frame of a point cloud to which the patch belongs. A non-matching patch is a patch for which no matching patch is found in the reference frame of the point cloud to which the patch belongs. The method for finding the matching patch is not limited in the present application.

The point cloud to which the matching patch belongs and the reference frame thereof may belong to the same subgroup (for example, the reference patch of the patch in the non-top frame point cloud in the 2 nd subgroup may be the patch in the same subgroup), or may belong to different subgroups (for example, the reference patch of the patch in the top frame (i.e., the 6 th frame) point cloud in the 2 nd subgroup may be the patch of the point cloud frame (the tail frame) in the 0 th subgroup or the 1 st subgroup).

A global match patch within a subgroup refers to a patch that has a matching relationship across all point clouds in a subgroup (also referred to as within a subgroup). For example, assuming a subset of 4 frames of point clouds (labeled point clouds 1-4, respectively), for any patch in point cloud 1, if a patch with a matching relationship to the patch can be found in point clouds 2-4, the following are the respective patches with a matching relationship to patch11 in point cloud 1: patch21 in point cloud 2, patch31 in point cloud 3, and patch41 in point cloud 4, then, patch11, 21, 31, 41 are all global matching patches in this sub-group, and patch q1 is global matching patch in current point cloud frame q, q is more than or equal to 1 and less than or equal to 4, q is an integer.

In contrast, a non-global match patch within a subgroup refers to a patch having a matching relationship across partial point clouds in a subgroup (also referred to as within a subgroup), or a non-global match patch is a patch for which no matching patch is found in all point clouds of the subgroup.

The global match patch identification for a patch indicates whether the patch is a global match patch within the subgroup. In the present application, the identification of the global match patch of the patch may be a first value (such as tune or 1) or a second value (such as false or 0). Wherein the first value is used to indicate that the patch is a global match patch within the subgroup; the second value is used to indicate that the patch is a non-global match patch within the subgroup.

The matching patch index of a patch is the index of the patch that matches the patch in the reference frame of the point cloud to which the patch belongs. The patch in the reference frame indicated by the matching patch index of the patch is the reference patch.

The match patch index of a patch may be a non-default value (e.g., 1,2,3, etc.) or a default value (e.g., -1). The matching patch index is a non-default value to indicate the patch (i.e. reference patch) which is matched with the patch in the reference frame of the point cloud to which the patch belongs, and accordingly, the patch is the matching patch; the matching patch index is a default value to indicate that the patch has no matching patch in the reference frame of the point cloud to which it belongs, and accordingly, the patch is a non-matching patch.

The two-dimensional information of the patch comprises u0, v0, sizeU0, sizeV0 and u0, wherein the abscissa value of the position of the occupancy map of the patch in the point cloud occupancy map to which the occupancy map belongs, and v0 is the ordinate value of the position of the occupancy map of the patch in the point cloud occupancy map to which the occupancy map of the patch belongs; sizeU0 is the width of the patch's occupancy graph; sizeV0 is the height of the patch occupying the graph.

The three-dimensional information for the patch includes u1, d1, and v1, u1 (also called patch _3d _ shift _ vary _ axis) is the offset of the point in the patch along the tangent axis, v1 (also called patch _3d _ shift _ bitangle _ axis) is the offset of the point in the patch along the bisector axis, and d1 (also called patch _3d _ shift _ normal _ axis) is the offset of the point in the patch along the normal axis.

The rotation transformation information of patch may be rotation angle information, coordinate axis transformation information, or mirror transformation information. Wherein the rotation angle information includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch is 0, it indicates that the patch is not rotated; if the value of the rotation angle of the patch is 20, the patch is rotated by 20 degrees; if the value of the rotation angle of the patch is 80, it indicates that the patch rotates 80 °. For another example, if the rotation angle index of the patch is 0, it indicates that the patch does not rotate; if the rotation angle index of the patch is 1, the patch is rotated by 15 degrees; if the rotation angle index of the patch is 2, the patch is rotated by 30 degrees; if the rotation angle index of the patch is 3, it indicates that the patch is rotated by 45 °.

The rotation of the patch may be clockwise or counterclockwise.

The coordinate axis exchange information may be represented as a coordinate axis exchange identifier, for example, when the coordinate axis exchange identifier is false, the tangent coordinate axis of the patch and the double tangent coordinate axis of the patch are not exchanged; for example, when the axis exchange flag is "tune", the tangent axis of patch is exchanged with the two-tangent axis of patch. For example, assuming that the U-axis index is 0 and the V-axis index is 1, the index of the tangent coordinate axis of the patch is 0 by default, and the tangent coordinate axis of the patch is described as the U-axis; the index of the two-tangent coordinate axis of patch is 1, which indicates that the two-tangent coordinate axis of patch is the V axis. If the coordinate axis exchange identification is false, the index of the tangent coordinate axis of the patch and the index of the double tangent coordinate axis of the patch are not exchanged; if the coordinate axis exchange mark is tune, the index of the tangent coordinate axis of the patch and the index of the double tangent coordinate axis of the patch are exchanged, that is, the index of the tangent coordinate axis of the patch is updated to 1, and the index of the double tangent coordinate axis of the patch is updated to 0.

Referring to fig. 6, fig. 6 is a schematic flowchart of a point cloud encoding method according to an embodiment of the present disclosure. As shown in fig. 6, the method includes:

s601, side information of the patch to be coded is obtained.

S602, if the patch to be coded is the patch in the first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream; the side information includes two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification, and matching patch index.

Wherein, the first frame point cloud of the subgroup refers to the 0 th frame point cloud in the subgroup.

S603, if the patch to be coded is the patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, wherein the side information comprises three-dimensional information and a matching patch index.

The non-first frame point cloud of the subgroup refers to the ith frame point cloud in the subgroup, i is 1,2, …, N-1, and N is the number of frames of the point cloud in the subgroup.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch and has a patch matched with the to-be-coded patch, the side information of the to-be-coded patch is coded into the code stream, the matching patch index of the to-be-coded patch is a non-default value, and the global matching patch index is a first value.

The matching patch index of the patch to be encoded is a non-default value, and the global matching patch is identified as a first value.

Optionally, the two-dimensional information and the three-dimensional information of the to-be-encoded patch may be encoded into the code stream by using a differential encoding method, and the rotation transformation information, the global matching patch identifier, and the matching patch index of the to-be-encoded patch may be encoded into the code stream by using a fixed-length encoding method.

If the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, including: if the to-be-encoded patch is a patch in the non-first frame point cloud in the subgroup, the to-be-encoded patch is a global matching patch and has a patch matched with the to-be-encoded patch, encoding side information of the to-be-encoded patch into the code stream, wherein a matching patch index of the to-be-encoded patch is a non-default value, and a global matching patch index is a first value.

Optionally, the three-dimensional information of the patch to be encoded may be encoded into the code stream by using a differential encoding method. Reference is made in particular to the above description of the related matters, which will not be further described here.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-encoded patch is a patch in the first frame point cloud in the subgroup, the to-be-encoded patch is a globally matched patch and does not have a patch matched with the to-be-encoded patch, the side information of the to-be-encoded patch further comprises normal coordinate axis information, and the side information of the to-be-encoded patch is encoded into the code stream. Normal axis coordinate information such as normalAxis, the normal axis index of the patch to be encoded.

The matching patch index of the to-be-coded patch is a default value, and the global matching patch identifier is a first value.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-coded patch is the patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch, and the to-be-coded patch has the patch matched with the to-be-coded patch, encoding the side information of the to-be-coded patch into the code stream.

The matching patch index of the patch to be coded is a non-default value, and the global matching patch identifier is a second value.

Optionally, the two-dimensional information and the three-dimensional information of the patch to be encoded may be encoded into the code stream by using a differential encoding method.

If the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, including: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch, the to-be-coded patch has a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, and the side information of the to-be-coded patch is coded into the code stream.

Optionally, a differential coding mode may be adopted to encode the two-dimensional information and the three-dimensional information of the patch to be coded into the code stream.

In one possible embodiment, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encoding the edge information of the patch to be encoded into the code stream, including: if the to-be-encoded patch is the patch in the first frame point cloud in the subgroup, the to-be-encoded patch is a non-global matching patch and does not have a patch matched with the to-be-encoded patch, the side information of the to-be-encoded patch further comprises normal coordinate axis information, and the side information of the to-be-encoded patch is encoded into the code stream. The matching patch index of the patch to be coded is a default value, and the global matching patch is marked as a second value;

if the patch to be coded is a patch in the non-first frame point cloud in the subgroup, encoding the side information of the patch to be coded into a code stream, including: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, normal coordinate axis information and rotation transformation information, and the side information of the to-be-coded patch is coded into the code stream.

The matching patch index of the patch to be coded is a default value, and the global matching patch identifier is a second value.

It should be noted that encoding two-dimensional information into the code stream by using a differential encoding method specifically includes encoding a difference between u0 of the patch to be encoded and u0 of the reference patch thereof, a difference between v0 of the patch to be encoded and v0 of the reference patch thereof, a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch thereof, and a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch thereof into the code stream; similarly, encoding the three-dimensional information into the code stream by using a differential encoding method specifically includes encoding a difference between u0 of the patch to be encoded and u0 of the reference patch, a difference between v0 of the patch to be encoded and v0 of the reference patch, a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch, and a difference between sizeU0 of the patch to be encoded and sizeU0 of the reference patch into the code stream. The differential coding mode can further achieve the purpose of reducing the bit overhead of the code stream.

In a possible embodiment, if the frame number of the point clouds in the subgroup is greater than 1, encoding a global matching patch identifier of a to-be-encoded patch in the point cloud of the first frame in the subgroup into a code stream; if the frame number of the point cloud in the subgroup is equal to 1, the global matching patch identification of the patch to be coded is not coded into the code stream.

In another possible embodiment, if the frame number of the point clouds in the subgroup is greater than 1, encoding the rotation transformation information of the to-be-encoded patch in the point cloud of the first frame in the subgroup into a code stream; if the frame number of the point cloud in the subgroup is equal to 1 and the patch to be coded is a matching patch, not coding the rotation transformation information of the patch to be coded into a code stream; if the frame number of the point cloud in the subgroup is equal to 1 and the patch to be coded is a non-matching patch, encoding the rotation transformation information of the patch to be coded into the code stream.

In a possible embodiment, if the patch to be encoded is a matching patch, the matching patch index of the patch to be encoded is encoded into the code stream; and if the patch to be coded is a non-matching patch, not coding the matching patch index of the patch to be coded into the code stream.

In one possible embodiment, the partial side information of the global match patch within the subgroup, which refers to the side information that is common to the global match patches and has the same value, is encoded separately. The partial side information refers to a part or all of the two-dimensional information, the normal coordinate axis information, and the rotational transform information. When encoding the side information of the patch of all the frames in the subgroup, if one patch in the point cloud of one frame in the subgroup is a global matching patch, part of the side information (i.e. part or all of the two-dimensional information, the normal coordinate axis information and the rotation transformation information) of the patch does not need to be encoded. The embodiment also achieves the purpose of improving the coding efficiency and simultaneously reduces the bit overhead of the code stream.

It can be seen that, in the scheme of the embodiment of the application, for the global matching patch, since partial side information of the global matching patch is the same, it is not necessary to encode partial side information of the global matching patch in the non-leading frame point cloud in the subgroup into the code stream. By adopting the method and the device, the matching relation that the rotation transformation information used for restricting the matching patch is consistent among the subgroups is abandoned, so that the packing of each subgroup is more flexible, and the sizes of the generated depth map and the texture map are smaller. Because partial side information of the global matching patch is coded only once, the purpose of improving coding efficiency is achieved, and bit overhead of the code stream is reduced.

Referring to fig. 7, fig. 7 is a schematic flowchart of a point cloud decoding method according to an embodiment of the present disclosure. As shown in fig. 7, the method includes:

s701, analyzing the side information of the patch to be decoded from the code stream.

S702a, if the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index, obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index of the patch to be decoded;

s702b, if the side information comprises three-dimensional information and a matching patch index, obtaining a global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch index of the patch to be decoded.

In a possible embodiment, if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, obtaining normal coordinate axis information of the to-be-decoded patch according to the matching patch index of the to-be-decoded patch, including: and if the matching patch index of the to-be-decoded patch is a non-default value, determining the normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index.

Specifically, the normal coordinate axis information of the reference patch indicated by the matching patch index is taken as the normal coordinate axis information of the patch to be decoded. For example, patch [ k ]. normalAxis ═ frame. refPatchId ═ patch [ k ]. matchedPatchIndex. Wherein, patch [ k ] is the patch to be decoded, reframe is the reference frame of the point cloud to which the patch belongs, and patch [ k ]. matchedPatchIndex is the matching patch index of the patch to be decoded, refPatchId is the index of the reference patch.

Optionally, the two-dimensional information and the three-dimensional information of the patch to be decoded may be parsed from the code stream in a differential decoding manner.

If the side information includes three-dimensional information and matching patch indexes, global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded are obtained according to the matching patch indexes of the patch to be decoded, and the method comprises the following steps of: if the matching patch index of the to-be-decoded patch is a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index. Specifically, the global matching patch identification of the reference patch indicated by the matching patch index is taken as the global matching patch identification of the patch to be decoded. For example, patch [ k ]. globalPatchFlag ═ frame. The globalpatchglag identifies a globally matching patch identification. If the global matching patch identifier of the to-be-decoded patch is a first value, respectively determining the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the to-be-decoded patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch indicated by the matching patch index. Specifically, the two-dimensional information, the normal coordinate axis information, and the rotational transform information of the reference patch indicated by the matching patch index are taken as the two-dimensional information, the normal coordinate axis information, and the rotational transform information of the patch to be decoded, respectively. For example:

patch[k].u0＝reframe.patch[refPatchId].u0，

patch[k].v0＝reframe.patch[refPatchId].v0，

patch[k].sizeU0＝reframe.patch[refPatchId].sizeU0，

patch[k].sizeV0＝reframe.patch[refPatchId].sizeV0，

patch[k].normalAxis＝reframe.patch[refPatchId].normalAxis，

patch[k].orientationIndex＝reframe.patch[refPatchId].orientationIndex。

wherein u0, v0, sizeU0 and sizeV0 are collectively called two-dimensional information, normalAxis represents normal coordinate axis information, and orientation index represents rotational transformation information.

Optionally, the three-dimensional information of the patch to be decoded is analyzed from the code stream by adopting a differential decoding mode.

In a possible embodiment, if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index, obtaining normal coordinate axis information of the to-be-decoded patch according to the matching patch index of the to-be-decoded patch, including: and if the matching patch index of the to-be-decoded patch is a default value, analyzing the normal coordinate axis information of the to-be-decoded patch from the code stream.

In a possible embodiment, if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index, obtaining normal coordinate axis information of the to-be-decoded patch according to the matching patch index of the to-be-decoded patch, including: and if the matching patch index of the to-be-decoded patch is a non-default value, determining the normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index. Specifically, the normal coordinate axis information of the reference patch indicated by the matching patch index is taken as the normal coordinate axis information of the patch to be decoded. For example:

patch[k].normalAxis＝reframe.patch[refPatchId].normalAxis。

refPatchId＝patch[k].matchedPatchIndex。

If the side information includes three-dimensional information and matching patch indexes, global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded are obtained according to the matching patch indexes of the patch to be decoded, and the method comprises the following steps of: if the matching patch index of the to-be-decoded patch is a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index. Specifically, the global matching patch identification of the reference patch indicated by the matching patch index is taken as the global matching patch identification of the patch to be decoded. For example, patch [ k ]. globalPatchFlag ═ frame. The globalPatchFlag identifies a global match patch identification. If the global matching patch identifier of the to-be-decoded patch is a second value, analyzing two-dimensional information of the to-be-decoded patch from the code stream, and respectively determining normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the normal coordinate axis information and the rotation transformation information of the reference patch indicated by the matching patch index. Specifically, the normal coordinate axis information and the rotational transform information of the reference patch indicated by the matching patch index are taken as the normal coordinate axis information and the rotational transform information of the patch to be decoded, respectively. For example:

patch[k].normalAxis＝reframe.patch[refPatchId].normalAxis，

patch[k].orientationIndex＝reframe.patch[refPatchId].orientationIndex。

optionally, a differential decoding mode can be adopted to analyze the two-dimensional information and the three-dimensional information of the patch to be decoded from the code stream.

If the side information includes three-dimensional information and a matching patch index, obtaining a global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch index of the patch to be decoded, including: and if the matching patch index of the to-be-decoded patch is a default value, analyzing the two-dimensional information, the normal coordinate axis information and the rotation transformation information from the code stream, and determining that the global matching patch identifier of the to-be-decoded patch is a second value. The purpose of determining the global matching patch identifier of the to-be-decoded patch as the second value is to directly determine the global matching patch identifier of the to-be-decoded patch as the global matching patch identifier of the to-be-decoded patch when the to-be-decoded patch is taken as the patch of the reference patch, so that the decoding is performed according to the global matching patch identifier.

It should be noted that, in the differential decoding, the two-dimensional information of the to-be-decoded patch is analyzed from the codestream, specifically, the difference between u0 of the to-be-decoded patch and u0 of the reference patch thereof, the difference between v0 of the to-be-decoded patch and v0 of the reference patch thereof, the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch thereof, and the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch thereof are analyzed from the codestream; then, u0 of the to-be-decoded patch is obtained according to the difference between u0 of the to-be-decoded patch and u0 of the reference patch and u0 of the reference patch, v0 of the to-be-decoded patch is obtained according to the difference between v0 of the to-be-decoded patch and v0 of the reference patch and v0 of the reference patch, sizeU0 of the to-be-decoded patch is obtained according to the difference between sizeU0 of the to-be-decoded patch and sizeU0 of the reference patch and sizeU0 of the reference patch, and sizeU0 of the to-be-decoded patch is obtained according to the difference between sizeU0 of the to-decoded patch and sizeU0 of the reference patch and sizeU0 of the reference patch. Similarly, the analyzing the three-dimensional information of the to-be-decoded patch from the code stream by adopting the differential decoding method specifically includes analyzing a difference value between u1 of the to-be-decoded patch and u1 of a reference patch thereof from the code stream, a difference value between v1 of the to-be-decoded patch and v1 of the reference patch thereof, and a difference value between d1 of the to-be-decoded patch and d1 of the reference patch thereof; then, u1 of the patch to be decoded is obtained according to the difference value between u1 of the patch to be decoded and u1 of the reference patch and u1 of the reference patch, v1 of the patch to be decoded is obtained according to the difference value between v1 of the patch to be decoded and v1 of the reference patch and v1 of the reference patch, and d1 of the patch to be decoded is obtained according to the difference value between d1 of the patch to be decoded and d1 of the reference patch and d1 of the reference patch.

In one possible embodiment, whether the to-be-decoded patch is the matching patch is determined by whether the matching patch index of the to-be-decoded patch is parsed from the codestream. And if the matching patch index of the to-be-decoded patch is obtained by analyzing the code stream, determining that the to-be-decoded patch is the matching patch, and if the matching patch index of the to-be-decoded patch is not obtained by analyzing the code stream, determining that the to-be-decoded patch is the non-matching patch.

In one possible embodiment, partial side information of the global match patch within the subgroup, which refers to side information that is common to the global match patch and has the same value, is decoded separately. The partial side information refers to a part or all of the two-dimensional information, the normal coordinate axis information, and the rotational transform information. When decoding the side information of the patches of all the frames in the subgroup, if one patch in a frame point cloud in the subgroup is a global matching patch, part of the side information (i.e., part or all of the two-dimensional information, the normal coordinate axis information, and the rotation transformation information) of the patch does not need to be decoded. This embodiment also achieves the purpose of improving the decoding efficiency.

S703, reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded.

It can be seen that, in the scheme of the embodiment of the application, the side information of the global matching patch in the sub-group head frame point cloud is analyzed from the code stream; because the partial side information of the global matching patch is the same, the partial side information of the global matching patch in the non-head frame point can be obtained by derivation directly according to the side information of the global matching patch in the head frame point cloud.

In one specific embodiment, as shown in FIG. 8, the reference frame of the 0 th sub-group of the I-th sub-group is the last frame point cloud in the I-1 th sub-group. The I subgroup comprises 3 frames of point clouds, namely a 0 th frame of point cloud, a 1 st frame of point cloud and a 2 nd frame of point cloud; each frame point cloud includes 9 patches. Wherein, in the reference frame of the 0 th frame point cloud in the I-th subgroup, the reference frame comprises a patch which is respectively matched with the 0 th, 1,2,3,4 and 5 patches in the 0 th frame point cloud in the I-th subgroup and a patch which is not matched with the 6 th, 7 th and 8 patches in the 0 th frame point cloud in the I-th subgroup. The 0,1,2,3,4,5 patches of the 0 th frame point cloud in the I subgroup are all matching patches; the 6 th, 7 th and 8 th patches are all non-matching patches; correspondingly, the matching patch indexes of 0,1,2,3,4,5 patches in the 0 th frame point cloud in the I th sub-group are 1,0,3,2,4,5, respectively.

The reference frame of the 1 st frame point cloud in the I subgroup is the 0 th frame point cloud in the I subgroup. In the reference frame of the 1 st frame point cloud in the I subgroup, a patch respectively matched with the 0,1,2,3,4,5,6,7 patches in the 1 st frame point cloud in the I subgroup and a patch not matched with the 8 th patch in the 1 st frame point cloud in the I subgroup are included. The 0 th, 1 st, 2 nd, 3 rd, 4 th, 5 th, 6 th, and 7 th patches of the 1 st frame point cloud in the I-th sub-group are all matching patches, and the 8 th patch is a non-matching patch. Accordingly, the matching patch indexes of 0,1,2,3,4,5,6,7 patches in the 1 st frame point cloud in the I-th sub-group are 0,2,1,4,3,5,6,7, respectively.

And the reference frame of the 2 nd frame point cloud in the I th subgroup is the 1 st frame point cloud in the I th subgroup. In the reference frame of the 2 nd frame point cloud in the I th subgroup, the reference frame comprises the patch which is respectively matched with the 0 th, 1 st, 2 nd, 3 th, 4 th and 5 th patches in the 1 st frame point cloud in the I th subgroup and the patches which are not matched with the 6 th, 7 th and 8 th patches in the 2 nd frame point cloud in the I th subgroup. The 0 th, 1 st, 2 nd, 3 th, 4 th, 5 th patches of the 2 nd frame point cloud in the I-th subgroup are matching patches, and the 6 th, 7 th, 8 th patches are non-matching patches. Accordingly, the matching patch indexes of the 0,1,2,3,4,5, and the patches in the 2 nd frame point cloud in the I th sub-group are 2,0,1,4,3, and 6, respectively. As shown in fig. 8, the gray boxes represent matching patches and the white boxes represent non-matching patches.

The 0,1,2,3,4,6 patches of the 0 th frame point cloud in the I subgroup, the 0,1,2,3,4,6 patches in the 1 st frame point cloud and the 0,1,2,3,4,5 patches in the 2 nd frame point cloud are all global matching patches; the 5 th, 7 th, 8 th patch of the 0 th frame point cloud in the I subgroup, the 5 th, 7 th, 8 th patch in the 1 st frame point cloud and the 6 th, 7 th, 8 th patch in the 2 nd frame point cloud are all non-global matching patches.

In encoding, for the 0 th frame point cloud in the I th sub-group: acquiring the two-dimensional information, the three-dimensional information, the rotation transformation information, the global matching patch identification and the matching patch index of each patch in the 0 th, 1,2,3,4,5 patches in the 0 th frame point cloud, and encoding the two-dimensional information, the three-dimensional information, the rotation transformation information, the global matching patch identification and the matching patch index of each patch in the 0 th, 1,2,3,4,5 patches into a code stream. And acquiring the two-dimensional information, the three-dimensional information, the normal coordinate axis information, the rotation transformation information and the global matching patch identifier of each patch in the 6 th, 7 th and 8 th frames of point cloud, and encoding the two-dimensional information, the three-dimensional information, the normal coordinate axis information, the rotation transformation information, the global matching patch identifier and the matching patch index into a code stream of each patch in the 6 th, 7 th and 8 th frames of point cloud.

For the 1 st frame point cloud in the ith subgroup: the method comprises the steps that all the 0 th, 1 st, 2 nd, 3 th, 4 th and 6 th patches in the 1 st frame point cloud are global matching patches, the three-dimensional information and matching patch indexes of all the 0 th, 1 st, 2 th, 3 th, 4 th and 6 th patches in the 1 st frame point cloud are obtained, and the three-dimensional information and matching patch indexes of all the 0 th, 1 st, 2 th, 3 th, 4 th and 6 th patches are coded into a code stream. Optionally, a differential coding mode is adopted to encode the difference value between the three-dimensional information of each patch of the 0 th, 1 st, 2 th, 3 th, 4 th, and 6 th patches and the three-dimensional information of the reference patch into the code stream. The method comprises the steps that the 5 th and 7 th patches in the 1 st frame point cloud are all matching patches and are non-global matching patches, two-dimensional information and three-dimensional information of each patch in the 5 th and 7 th patches in the 1 st frame point cloud are obtained, and the two-dimensional information and the three-dimensional information of each patch in the 5 th and 7 th patches are coded into a code stream. And (3) the 8 th patch in the 1 st frame point cloud is a non-matching patch, the two-dimensional information, the three-dimensional information, the normal coordinate axis information, the rotation transformation information and the matching patch index of the 8 th patch in the 1 st frame point cloud are obtained, and the two-dimensional information, the three-dimensional information, the normal coordinate axis information, the rotation transformation information and the matching patch index of the 8 th patch in the 1 st frame point cloud are coded into a code stream.

For the 2 nd frame point cloud in the I th subgroup: the method comprises the steps that all the 0 th, 1 st, 2 nd, 3 th, 4 th and 5 th patches in the 2 nd frame point cloud are global matching patches, the three-dimensional information and matching patch indexes of all the 0 th, 1 st, 2 th, 3 th, 4 th and 5 th patches in the 2 nd frame point cloud are obtained, and the three-dimensional information and matching patch indexes of all the 0 th, 1 st, 2 th, 3 th, 4 th and 5 th patches are coded into a code stream. The method comprises the steps of obtaining two-dimensional information, three-dimensional information, normal coordinate axis information, rotation transformation information and matching patch indexes of each patch in the 6 th, 7 th and 8 th frames of point cloud in the 2 nd frame of point cloud, and coding the two-dimensional information, the three-dimensional information, the normal coordinate axis information, the rotation transformation information and the matching patch indexes of each patch in the 6 th, 7 th and 8 th frames of point cloud in the 2 nd frame of point cloud.

When decoding, for the 0 th frame point cloud in the I subgroup: analyzing a matching patch index, two-dimensional information, three-dimensional information, rotation transformation information and a global matching patch identifier of each patch in the 0 th frame point cloud from the code stream; determining that the 0 th, 1 st, 2 nd, 3 th, 4 th and 5 th patches are all matching patches according to the matching patch indexes of each patch, determining that the 6 th, 7 th and 8 th patches are all non-matching patches, and determining the normal coordinate axis information of each of the 0 th, 1 st, 2 th, 3 th, 4 th and 5 th patches according to the normal coordinate axis information of the reference patch of each of the 0 th, 1 st, 2 th, 3 th, 4 th and 5 th patches; and analyzing the normal coordinate axis information of each of the 6 th, 7 th and 8 th latches from the code stream.

For the 1 st frame point cloud in the ith subgroup: analyzing the three-dimensional information of each patch in the 1 st frame point cloud and matching the patch index from the code stream; and determining that the 0 th, 1 st, 2 th, 3 th, 4 th, 5 th, 6 th and 7 th patches in the 1 st frame point cloud are all matching patches according to the matching patch indexes of each patch, and determining that the 8 th patch is a non-matching patch. And determining the global matching patch identification of each patch according to the global matching patch identification of the reference patch of each patch of 0,1,2,3,4,5,6 and 7 patches in the 1 st frame point cloud. According to the global matching patch identification of each patch of 0,1,2,3,4,5,6 and 7 patches, determining that the patch of 0,1,2,3,4 and 6 is a global matching patch, and determining that the patch of 5 and 7 is a non-global matching patch. Determining the two-dimensional information, the normal coordinate axis information and the rotation transformation information of each patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch of each patch in the 0 th, 1 st, 2 nd, 3 th, 4 th and 6 th patches respectively; analyzing the two-dimensional information of each of the 5 th and 7 th patches from the code stream, and respectively determining the normal coordinate axis information and the rotation transformation information of each of the 5 th and 7 th patches according to the normal coordinate axis information and the rotation transformation information of the reference patch of each of the 5 th and 7 th patches. And analyzing the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the 8 th patch from the code stream, and determining that the global matching patch of the 8 th patch is identified as a second value (such as 0 or false), wherein the second value is used for indicating that the 8 th patch is a non-global matching patch.

For the 2 nd frame point cloud in the I th subgroup: analyzing the three-dimensional information and the matching patch index of each patch in the 2 nd frame point cloud from the code stream; and determining that the 0 th, 1 st, 2 nd, 3 th, 4 th and 5 th patches in the 2 nd frame point cloud are all matching patches according to the matching patch index of each patch, and determining that the 6 th, 7 th and 8 th patches are all non-matching patches. And determining the global matching patch identification of each patch according to the global matching patch identification of the reference patch of each patch of 0,1,2,3,4,5 patches in the 2 nd frame point cloud. And respectively determining the 0 th, 1 st, 2 nd, 3 rd, 4 th and 5 th patch to be the global matching patch according to the global matching patch identification of each of the 0 th, 1 st, 2 th, 3 th, 4 th and 5 th patches. The two-dimensional information, the normal coordinate axis information and the rotational transformation information of each of the 0 th, 1 st, 2 nd, 3 th, 4 th, 5 th patches are determined from the two-dimensional information, the normal coordinate axis information and the rotational transformation information of the reference patch of the each of the patches. And analyzing the two-dimensional information, the normal coordinate axis information and the rotation transformation information of each of the 6 th, 7 th and 8 th patches from the code stream, and determining that the global matching patch of each of the 6 th, 7 th and 8 th patches is identified as a second value (such as 0 or false), wherein the second value is used for indicating that the patch is a non-global matching patch.

In a possible embodiment, an embodiment of the present application further provides a point cloud encoding method, including: acquiring side information of a patch to be coded; and encoding indication information into the code stream, wherein the indication information is used for indicating whether the side information of the patch to be encoded is encoded according to a target encoding method, and the target encoding method refers to part or all of the content of the embodiment shown in fig. 6, and is not described herein.

In one possible embodiment, an embodiment of the present application further provides a point cloud decoding method, including: analyzing the code stream to obtain indication information, wherein the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method is described in part or in whole with reference to the embodiment shown in fig. 7, and will not be described herein; and reconstructing the patch to be decoded according to the side information of the patch to be decoded.

In a possible embodiment, an embodiment of the present application further provides a point cloud decoding method, including: analyzing the code stream to obtain indication information, wherein the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method is described in part or in whole with reference to the embodiment shown in fig. 7, and will not be described herein; and reconstructing the patch to be decoded according to the side information of the patch to be decoded.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an apparatus for encoding a point cloud according to an embodiment of the present disclosure. As shown in fig. 9, the encoding apparatus 900 may include:

a cloud patch (patch) information obtaining module 901, configured to obtain side information of a patch to be encoded;

the auxiliary information encoding module 902 is configured to, if the patch to be encoded is a patch in the first frame point cloud in the subgroup, encode side information of the patch to be encoded into the code stream; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index; the global matching patch identifier is used for indicating whether the patch to be coded is the global matching patch in the subgroup or not;

the auxiliary information encoding module 902 is further configured to, if the patch to be encoded is a patch in a non-first-frame point cloud in the subgroup, encode edge information of the patch to be encoded into the code stream, where the edge information includes three-dimensional information and a matching patch index.

In one possible embodiment, in the aspect that if the patch to be encoded is a patch in the first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the code stream, the auxiliary information encoding module 902 is specifically configured to: if the to-be-encoded patch is the patch in the first frame point cloud in the subgroup, and the to-be-encoded patch is a global matching patch and has a patch matched with the global matching patch, encoding the side information of the to-be-encoded patch into the code stream.

In the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the non-first-frame point cloud in the subgroup, the auxiliary information encoding module 902 is specifically configured to: if the to-be-encoded patch is a patch in the non-first frame point cloud in the subgroup, and the to-be-encoded patch is a global matching patch and has a patch matched with the global matching patch, encoding the side information of the to-be-encoded patch into the code stream.

The matching patch index of the patch to be encoded is a non-default value to indicate a reference patch matching the patch to be encoded. The global match patch flag of the to-be-encoded patch is a first value to indicate that the to-be-encoded patch is a global match patch within the subgroup.

In a possible embodiment, in the aspect that if the patch to be encoded is a patch in the first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the code stream, the auxiliary information encoding module 902 is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a globally matched patch and the to-be-coded patch does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch also comprises normal coordinate axis information, and the side information of the to-be-coded patch is coded into a code stream; wherein the matching patch index is used to indicate that the patch to be encoded does not have a patch matching it. The matching patch index of the to-be-encoded patch is a default value to indicate that the to-be-encoded patch does not have a patch matching therewith; the global match patch flag of the to-be-encoded patch is a first value to indicate that the to-be-encoded patch is a global match patch within the subgroup.

In a possible embodiment, in the aspect that if the patch to be encoded is a patch in the first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the code stream, the auxiliary information encoding module 902 is specifically configured to: if the to-be-encoded patch is the patch in the first frame point cloud in the subgroup, the to-be-encoded patch is a non-global matching patch and has a patch matched with the to-be-encoded patch, encoding side information of the to-be-encoded patch into a code stream;

in the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the non-first-frame point cloud in the subgroup, the auxiliary information encoding module 902 is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch, the to-be-coded patch has a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, and the side information of the to-be-coded patch is coded into the code stream.

The matching patch index of the patch to be coded is a non-default value to indicate a reference patch matched with the patch to be coded; the global match patch flag of the to-be-encoded patch is a second value to indicate that the to-be-encoded patch is not a global match patch within the subgroup.

In a possible embodiment, in the aspect that if the patch to be encoded is a patch in the first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the code stream, the auxiliary information encoding module 902 is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and the to-be-coded patch does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch also comprises normal coordinate axis information, and the side information of the to-be-coded patch is coded into a code stream;

in the aspect of encoding the side information of the to-be-encoded patch into the code stream if the to-be-encoded patch is a patch in the non-first-frame point cloud in the subgroup, the auxiliary information encoding module 902 is specifically configured to: if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and does not have a patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, normal coordinate axis information and rotation transformation information, and the side information of the to-be-coded patch is coded into the code stream.

Wherein, the matching patch index of the patch to be coded is a default value to indicate that the patch to be coded does not have a patch matched with the matching patch; the global match patch flag of the to-be-encoded patch is a second value to indicate that the to-be-encoded patch is not a global match patch within the subgroup.

It should be noted that the patch information obtaining module 901 is configured to execute the relevant content of step S601 in the embodiment shown in fig. 6, and the auxiliary information encoding module 902 is configured to execute the relevant content of steps S602 and S603 in the embodiment shown in fig. 6. As an example, the patch information obtaining module 901 may correspond to a combination of the patch information generating module 101 and the packaging module 102 in fig. 2, and the auxiliary information encoding module 902 may correspond to the auxiliary information encoding module 108 in fig. 2, in other words, the functions of the patch information obtaining module 901 may be implemented by the patch information generating module 101 and the packaging module 102 in fig. 2, and the functions of the auxiliary information generating module 902 may be implemented by the auxiliary information encoding module 108 in fig. 2. In one example, the patch information generating module 101 is configured to obtain three-dimensional information, normal coordinate axis information, and global matching patch identification of the patch to be encoded, sizeU0 and sizeV0 in the two-dimensional information, and the packing module 102 is configured to obtain rotational transformation information of the patch to be encoded, matching patch index, and u0 and v0 in the two-dimensional information. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the encoder shown in fig. 2, which is not described here in detail.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a point cloud decoding apparatus according to an embodiment of the present disclosure. As shown in fig. 10, the decoding apparatus 1000 includes:

the auxiliary information decoding module 1001 is configured to parse side information of the patch to be decoded from the code stream;

if the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index, obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index;

if the side information comprises three-dimensional information and matching patch indexes, obtaining global matching patch identification, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch indexes of the patch to be decoded;

the reconstruction module 1002 is configured to reconstruct the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information, and the rotation transformation information of the patch to be decoded; the global matching patch identifier is used for indicating whether the to-be-decoded patch is the global matching patch or not.

In a possible embodiment, in the aspect that if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, the normal coordinate axis information of the patch to be decoded is obtained according to the matching patch index, the auxiliary information decoding module 1001 is specifically configured to:

and if the matching patch index of the to-be-decoded patch is determined to be a non-default value, determining the normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index.

If the side information includes three-dimensional information and a matching patch index, the auxiliary information decoding module 1001 specifically is configured to, according to the matching patch index of the to-be-decoded patch, obtain the global matching patch identifier, two-dimensional information, normal coordinate axis information, and rotation transformation information of the to-be-decoded patch: if the matching patch index of the to-be-decoded patch is determined to be a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch; if the global matching patch identifier of the to-be-decoded patch is a first value, determining the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the to-be-decoded patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch.

and if the matching patch index of the to-be-decoded patch is determined to be a default value, analyzing the coordinate axis information of the to-be-decoded patch from the code stream.

if the side information includes three-dimensional information and a matching patch index, the auxiliary information decoding module 1001 specifically is configured to, according to the matching patch index of the to-be-decoded patch, obtain the global matching patch identifier, two-dimensional information, normal coordinate axis information, and rotation transformation information of the to-be-decoded patch:

if the matching patch index of the to-be-decoded patch is determined to be a non-default value; determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is determined to be a second value; then, the two-dimensional information of the patch to be decoded is analyzed from the code stream; and normal coordinate axis information and rotational transformation information of the patch to be decoded are respectively determined according to the normal coordinate axis information and the rotational transformation information of the reference patch indicated by the matching patch index.

In a possible embodiment, in the aspect that if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier, and a matching patch index, the auxiliary information decoding module 1001 is specifically configured to, according to the matching patch index, obtain normal coordinate axis information of a patch to be decoded:

if the side information includes three-dimensional information and a matching patch index, the auxiliary information decoding module 1001 specifically is configured to, according to the matching patch index of the to-be-decoded patch, obtain a global matching patch identifier, two-dimensional information, normal coordinate axis information, and rotation transformation information of the to-be-decoded patch: and if the matching patch index of the to-be-decoded patch is determined to be a default value, analyzing the two-dimensional information, the normal coordinate axis information and the rotation transformation information from the code stream, and determining that the global matching patch identifier of the to-be-decoded patch is a second value.

It should be noted that the auxiliary information decoding module 1001 is configured to execute the relevant content in steps S701 and S702 in the embodiment shown in fig. 7, and the reconstruction module 1002 is configured to execute the relevant content in step S703 in the embodiment shown in fig. 7. As an example, the auxiliary information decoding module 1001 corresponds to the auxiliary information decoding module 204 in fig. 4, and the reconstruction module 1002 corresponds to the geometric information reconstruction module 205 of the point cloud in fig. 4, in other words, the function of the auxiliary information decoding module 1001 may be implemented by the auxiliary information decoding module 204 in fig. 4, and the function of the reconstruction module 1002 may be implemented by the geometric information reconstruction module 205 of the point cloud in fig. 4. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in fig. 4, which is not described here in detail.

Referring to fig. 11, fig. 11 is a schematic structural diagram of an apparatus for encoding a point cloud according to an embodiment of the present disclosure. As shown in fig. 11, the encoding apparatus 1100 includes:

a patch information obtaining module 1101, configured to obtain side information of the patch to be encoded;

an auxiliary information encoding module 1102, configured to encode indication information into the code stream, where the indication information is used to indicate whether to encode side information according to a target encoding method; the target encoding method includes part or all of the point cloud encoding method of the embodiment shown in fig. 6.

It should be noted that, as an example, the patch information obtaining module 1101 in fig. 11 corresponds to a combination of the patch information generating module 101 and the packing module 102 in fig. 2, and the auxiliary information encoding module 1102 in fig. 11 corresponds to the auxiliary information encoding module 108 in fig. 2, in other words, the function of the patch information obtaining module 1101 is implemented by the patch information generating module 101 and the packing module 102 in fig. 2, and the function of the auxiliary information encoding module 1102 may be implemented by the auxiliary information encoding module 108 in fig. 2. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the encoder shown in fig. 2, which is not described here in detail.

Referring to fig. 12, fig. 12 is a schematic structural diagram of another apparatus for decoding point clouds according to an embodiment of the present disclosure. As shown in fig. 12, the decoding apparatus 1200 includes:

an auxiliary information decoding module 1201, configured to parse the code stream to obtain indication information, where the indication information is used to indicate whether to decode the side information according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method includes part or all of the point cloud decoding method of the embodiment shown in fig. 7;

a reconstructing module 1202, configured to reconstruct the to-be-decoded patch according to the side information of the to-be-decoded patch.

It should be noted that, as an example, the auxiliary information decoding module 1201 in fig. 12 corresponds to the auxiliary information decoding module 204 in fig. 4, and the reconstruction module 1202 in fig. 12 corresponds to the geometric information reconstruction module 205 of the point cloud in fig. 4, in other words, the function of the auxiliary information decoding module 1201 may be implemented by the auxiliary information decoding module 204 in fig. 4, and the function of the reconstruction module 1202 may be implemented by the geometric information reconstruction module 205 of the point cloud in fig. 4. Although the application is not so limited. For further details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in fig. 4, which is not described here in detail.

FIG. 13 is a schematic block diagram of one implementation of a decoding device 1300 for an embodiment of the present application. The obtaining apparatus 1300 may include a processor 1301, a memory 1302, and a bus system 1303, among others. The processor 1301 and the memory 1302 are connected through a bus system 1303, the memory 1302 is used for storing instructions, and the processor 1301 is used for executing the instructions stored in the memory 1302 to execute various point cloud encoding or decoding methods described herein, especially a method for filtering a current image block based on a block size of the current image block. To avoid repetition, it is not described in detail here.

In this embodiment of the application, the processor 1301 may be a Central Processing Unit (CPU), and the processor 1301 may also be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 1302 may include a ROM device or a RAM device. Any other suitable type of memory device can also be used for memory 1302. The memory 1302 may include code and data 13021 accessed by the processor 1301 using the bus 1303. The memory 1302 may further include an operating system 13023 and application programs 13022, the application programs 13022 including at least one program that allows the processor 1301 to perform the point cloud encoding or decoding methods described herein (and in particular the point cloud encoding or decoding methods described herein). For example, application 13022 may include applications 1 through N, which further include point cloud encoding or decoding applications that perform the point cloud encoding or decoding methods described herein (referred to as point cloud decoding applications).

The bus system 1303 may include a power bus, a control bus, a status signal bus, and the like, in addition to the data bus. But for clarity of illustration the various busses are labeled in the figure as the bus system 1303.

Optionally, the encoding apparatus 1300 may also include one or more output devices, such as a display 1304. In one example, the display 1304 may be a touch-sensitive display that incorporates a display with a touch-sensing unit operable to sense touch input. A display 1304 may be connected to the processor 1301 via the bus 1303.

It should be noted that the decoding apparatus 1300 may perform the method for encoding a point cloud in the present application, and may also perform the method for decoding a point cloud in the present application.

Those of skill in the art will appreciate that the functions described in connection with the various illustrative logical blocks, modules, and algorithm steps described in the disclosure herein may be implemented as hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logical blocks, modules, and steps may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or any communication medium including a medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol). In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described herein. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory tangible storage media. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, DVD and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, Application Specific Integrated Circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor," as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements. In one example, various illustrative logical blocks, units, and modules within the encoder 100 and the decoder 200 may be understood as corresponding circuit devices or logical elements.

The techniques of this application may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an Integrated Circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described herein to emphasize functional aspects of means for performing the disclosed techniques, but do not necessarily require realization by different hardware units. Indeed, as described above, the various units may be combined in a codec hardware unit, in conjunction with suitable software and/or firmware, or provided by an interoperating hardware unit (including one or more processors as described above).

The above description is only an exemplary embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A point cloud encoding method, comprising:

acquiring side information of a cloud block patch to be coded;

if the patch to be coded is the patch in the first frame point cloud in the subgroup, coding the side information of the patch to be coded into a code stream; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch indexes; the global matching patch identifier is used for indicating whether the to-be-coded patch is a global matching patch in the subgroup or not;

if the to-be-encoded patch is the patch in the non-first frame point cloud in the subgroup, encoding side information of the to-be-encoded patch into the code stream, wherein the side information comprises three-dimensional information and a matching patch index.

2. The method according to claim 1, wherein if the patch to be encoded is a patch in a first frame point cloud in a subgroup, encoding side information of the patch to be encoded into a code stream, including:

if the to-be-coded patch is a patch in the subgroup initial frame point cloud, the to-be-coded patch is a global matching patch and has a reference patch matched with the to-be-coded patch, the edge information of the to-be-coded patch is coded into the code stream, the matching patch index is a non-default value to indicate the reference patch matched with the to-be-coded patch, and the global matching patch identifier is a first value to indicate the to-be-coded patch to be a global matching patch in the subgroup;

if the to-be-coded patch is the patch in the first frame point cloud in the subgroup, encoding the side information of the to-be-coded patch into the code stream, including:

if the to-be-coded patch is the patch in the subgroup non-first frame point cloud, the to-be-coded patch is a global matching patch and has a reference patch matched with the to-be-coded patch, the edge information of the to-be-coded patch is coded into the code stream, and the matching patch index is a non-default value to indicate the reference patch matched with the to-be-coded patch.

3. The method according to claim 1, wherein if the patch to be encoded is a patch in a first frame point cloud in a subgroup, encoding side information of the patch to be encoded into a code stream, including:

if treat the coding patch and do the patch in the first frame point cloud in the subgroup, just treat the coding patch and match the patch just for the whole situation the waiting coding patch does not have assorted reference patch with it, the limit information of treating the coding patch still includes normal coordinate axis information, will the limit information of treating the coding patch is compiled the code stream, the matching patch index is the default, in order to indicate the waiting coding patch does not have assorted reference patch with it, the whole situation matches the patch sign and is first value, in order to indicate the waiting coding patch is the whole situation in the subgroup matches the patch.

4. The method of claim 1, wherein if the patch to be encoded is a patch in a first frame point cloud in a subgroup, encoding edge information of the patch to be encoded into a code stream, comprising:

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch and has a reference patch matched with the to-be-coded patch, encoding edge information of the to-be-coded patch into the code stream, wherein the matching patch index is a non-default value to indicate the reference patch matched with the to-be-coded patch, and the global matching patch identifier is a second value to indicate that the to-be-coded patch is not the global matching patch in the subgroup;

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a non-global matching patch, the to-be-coded patch has a reference patch matched with the to-be-coded patch, the side information of the to-be-coded patch further comprises two-dimensional information, and the side information of the to-be-coded patch is coded into the code stream; wherein the matching patch index is a non-default value to indicate a reference patch that matches the patch to be encoded.

5. A point cloud decoding method, comprising:

analyzing the side information of the point cloud block patch to be decoded from the code stream;

if the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and a matching patch index of the to-be-decoded patch, obtaining normal coordinate axis information of the to-be-decoded patch according to the matching patch index;

if the side information comprises the three-dimensional information of the patch to be decoded and the matching patch index, obtaining a global matching patch identifier, two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the matching patch index;

reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded;

wherein the global match patch flag is used to indicate whether the to-be-decoded patch is a global match patch within the subgroup.

6. The method of claim 5, wherein obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index comprises:

if the matching patch index is determined to be a non-default value, determining normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index;

the basis match the patch index and obtain wait to decode the global match patch sign, two-dimensional information, normal coordinate axis information and the rotation transform information of patch, include:

if the matching patch index is determined to be a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is a first value, respectively determining the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the to-be-decoded patch according to the two-dimensional information, the normal coordinate axis information and the rotation transformation information of the reference patch;

wherein, the matching patch index is a non-default value to indicate with the reference patch matched with the to-be-decoded patch, the global matching patch identifier is a first value to indicate that the to-be-decoded patch is the global matching patch in the subgroup.

7. The method of claim 5, wherein obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index comprises:

if the matching patch index is determined to be a default value, normal coordinate axis information of the patch to be decoded is analyzed from the code stream;

wherein the match patch index is a default value to indicate that the to-be-decoded patch has no reference patch that matches it.

8. The method of claim 5, wherein the obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index comprises:

the basis the match patch index obtains wait to decode the global match patch sign, two-dimensional information, normal coordinate axis information and the rotation transform information of patch, include:

if the matching patch index is determined to be a non-default value, determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is a second value, analyzing the two-dimensional information of the to-be-decoded patch from the code stream, and respectively determining the normal coordinate axis information and the rotation transformation information of the to-be-decoded patch according to the normal coordinate axis information and the rotation transformation information of the reference patch;

wherein the match patch index is a non-default value to indicate a reference patch that matches the to-be-decoded patch, and the global match patch flag is a second value to indicate that the to-be-decoded patch is not a global match patch within the subset.

9. The method of claim 5, wherein the obtaining normal coordinate axis information of the patch to be decoded according to the matching patch index comprises:

if the matching patch index is determined to be a default value, analyzing the two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch from the code stream, and determining that the global matching patch identifier of the to-be-decoded patch is a second value;

wherein the match patch index is a default value to indicate that the to-be-decoded patch has no reference patch that matches it, the global match patch flag is a second value to indicate that the to-be-decoded patch is not a global match patch within the subgroup.

10. A point cloud encoding method, comprising:

acquiring side information of a patch to be coded;

encoding indication information into a code stream, wherein the indication information is used for indicating whether side information of the patch to be encoded is encoded according to a target encoding method; the object encoding method comprises the point cloud encoding method of any of claims 1-4.

11. A point cloud decoding method, comprising:

analyzing the code stream to obtain indication information, wherein the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; the target decoding method comprises the point cloud decoding method of any of claims 5-9;

and reconstructing the patch to be decoded according to the side information of the patch to be decoded.

12. An apparatus for encoding a point cloud, comprising:

the auxiliary information encoding module is used for encoding the side information of the to-be-encoded patch into a code stream if the to-be-encoded patch is the patch in the first frame point cloud in the subgroup; the side information comprises two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index; the global matching patch identifier is used for indicating whether the to-be-coded patch is a global matching patch in the subgroup or not;

the auxiliary information coding module is also used for encoding the patch to be encoded into the code stream if the patch to be encoded is the patch in the non-first frame point cloud in the subgroup, wherein the side information of the patch to be encoded is encoded into the code stream and comprises three-dimensional information and a matching patch index.

13. The apparatus of claim 12, wherein in the aspect that if the patch to be encoded is a patch in a first frame point cloud in a subgroup, encoding edge information of the patch to be encoded into a code stream, the auxiliary information encoding module is specifically configured to:

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch, and the to-be-coded patch has a patch matched with the global matching patch, encoding edge information of the to-be-coded patch into the code stream, wherein the matching patch index is a non-default value to indicate a reference patch matched with the to-be-coded patch; the global matching patch identifier is a first value to indicate that the to-be-encoded patch is a global matching patch within the subgroup;

if the to-be-encoded patch is a patch in the non-top frame point cloud in the subgroup, encoding side information of the to-be-encoded patch into the aspect of the code stream, wherein the auxiliary information encoding module is specifically configured to:

if wait to encode the patch and be in the patch in the non first frame point cloud in the subgroup, just wait to encode the patch and match the patch for the global just wait to encode the patch has assorted patch with it, will wait to encode the side information coding of patch the code stream, the matching patch index is non-default, with indicate with wait to encode the reference patch of patch assorted.

14. The apparatus according to claim 12, wherein in the aspect that if the patch to be encoded is a patch in a first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the codestream, the auxiliary information encoding module is specifically configured to:

if the to-be-coded patch is a patch in the first frame point cloud in the subgroup, the to-be-coded patch is a global matching patch and the to-be-coded patch does not have a patch matched with the global matching patch, the side information of the to-be-coded patch further comprises normal coordinate axis information, the side information of the to-be-coded patch is coded into the code stream, and the matching patch index is a default value to indicate that the to-be-coded patch does not have a patch matched with the matching patch; the global matching patch identifier is a first value to indicate that the to-be-encoded patch is a global matching patch within the subgroup.

15. The apparatus according to claim 12, wherein in the aspect that if the patch to be encoded is a patch in a first frame point cloud in the subgroup, the side information of the patch to be encoded is encoded into the codestream, the auxiliary information encoding module is specifically configured to:

if the to-be-encoded patch is a patch in the initial frame point cloud in the subgroup, the to-be-encoded patch is a non-global matching patch and has a patch matched with the to-be-encoded patch, encoding edge information of the to-be-encoded patch into the code stream, and the matching patch index is a non-default value to indicate a reference patch matched with the to-be-encoded patch; the global match patch flag is a second value to indicate that the to-be-encoded patch is not a global match patch within the subgroup;

if the to-be-encoded patch is a patch in the non-first-frame point cloud in the subgroup, encoding side information of the to-be-encoded patch into the aspect of the code stream, wherein the auxiliary information encoding module is specifically configured to:

if the to-be-coded patch is the patch in the subgroup non-first frame point cloud, and the to-be-coded patch is non-global matching patch and the to-be-coded patch has a matched patch therewith, the edge information of the to-be-coded patch further comprises two-dimensional information, the edge information of the to-be-coded patch is coded into the code stream, and the matching patch index is a non-default value to indicate and the to-be-coded patch matched reference patch.

16. A point cloud decoding apparatus, comprising:

the auxiliary information decoding module is used for analyzing the side information of the point cloud block patch to be decoded from the code stream;

the auxiliary information decoding module is further configured to obtain normal coordinate axis information of the patch to be decoded according to the matching patch index if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, a global matching patch identifier and the matching patch index; if the side information comprises three-dimensional information and a matching patch index, obtaining a global matching patch identifier, two-dimensional information, normal coordinate axis information and rotation transformation information of the to-be-decoded patch according to the matching patch index;

the reconstruction module is used for reconstructing the patch to be decoded according to the two-dimensional information, the three-dimensional information, the normal coordinate axis information and the rotation transformation information of the patch to be decoded;

wherein the global matching patch identification is used for indicating whether the to-be-decoded patch is the global matching patch in the subgroup.

17. The apparatus according to claim 16, wherein in the aspect of obtaining the normal coordinate axis information of the to-be-decoded patch according to the matching patch index, the auxiliary information decoding module is specifically configured to:

if the matching patch index is determined to be a non-default value, obtaining normal coordinate axis information of the to-be-decoded patch according to the normal coordinate axis information of the reference patch indicated by the matching patch index;

the auxiliary information decoding module is specifically configured to, according to the matching patch index, obtain the global matching patch identifier, the two-dimensional information, the normal coordinate axis information, and the rotation transformation information of the to-be-decoded patch, specifically:

wherein, the match patch index is a default value to indicate the reference patch of the to-be-decoded patch, and the global match patch identifier is a first value to indicate that the to-be-decoded patch is the global match patch in the subgroup.

18. The apparatus of claim 16, wherein in the aspect of obtaining the normal coordinate axis information of the patch to be decoded according to the matching patch index, the auxiliary information decoding module is specifically configured to:

wherein, the matching patch index is a default value to indicate that the to-be-decoded patch does not have a matching patch, and the global matching patch identifier is a first value to indicate that the to-be-decoded patch is the global matching patch in the subgroup.

19. The apparatus of claim 16, wherein in the aspect of obtaining the normal coordinate axis information of the patch to be decoded according to the matching patch index, the auxiliary information decoding module is specifically configured to:

if the matching patch index of the to-be-decoded patch is determined to be a non-default value; determining a global matching patch identifier of the to-be-decoded patch according to the global matching patch identifier of the reference patch indicated by the matching patch index; if the global matching patch identifier of the to-be-decoded patch is determined to be a second value; analyzing the two-dimensional information of the patch to be decoded from the code stream; respectively determining normal coordinate axis information and rotation transformation information of the patch to be decoded according to the normal coordinate axis information and the rotation transformation information of the reference patch;

wherein the match patch index is a non-default value to indicate the reference patch of the to-be-decoded patch, and the global match patch flag is a second value to indicate that the to-be-decoded patch is not a global match patch within the subset.

20. The apparatus of claim 16, wherein in the aspect of determining the normal coordinate axis information of the to-be-decoded patch according to the matching patch index, the auxiliary information decoding module is specifically configured to:

in aspects of determining the global matching patch identifier, the two-dimensional information, the normal coordinate axis information, and the rotation transformation information of the to-be-decoded patch according to the matching patch index, the auxiliary information decoding module is specifically configured to:

wherein the match patch index is a default value to indicate that the to-be-decoded patch has no patch matching therewith, and the global match patch flag is a second value to indicate that the to-be-decoded patch is not the global match patch within the subgroup.

21. An apparatus for encoding a point cloud, comprising:

the auxiliary information coding module is used for coding indication information into a code stream, wherein the indication information is used for indicating whether the side information of the patch to be coded is coded according to a target coding method; the object encoding method comprises the point cloud encoding method as claimed in any one of claims 1 to 4.

22. An apparatus for decoding a point cloud, comprising:

the auxiliary information decoding module is used for analyzing the code stream to obtain indication information, and the indication information is used for indicating whether the side information is decoded according to a target decoding method; when the indication information is used for indicating that the side information is decoded according to a target decoding method, obtaining the side information of the patch to be decoded according to the target decoding method; the target decoding method comprises the point cloud decoding method of any one of claims 5 to 9;

23. A computer-readable storage medium, characterized by comprising program code which, when run on a computer, causes the computer to perform the point cloud encoding method of any of claims 1 to 4 and 10.

24. A computer-readable storage medium, characterized by comprising program code which, when run on a computer, causes the computer to perform the point cloud decoding method of any of claims 5 to 9 and 11.