WO2020187283A1 - Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage - Google Patents

Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage Download PDF

Info

Publication number
WO2020187283A1
WO2020187283A1 PCT/CN2020/080140 CN2020080140W WO2020187283A1 WO 2020187283 A1 WO2020187283 A1 WO 2020187283A1 CN 2020080140 W CN2020080140 W CN 2020080140W WO 2020187283 A1 WO2020187283 A1 WO 2020187283A1
Authority
WO
WIPO (PCT)
Prior art keywords
patch
information
decoded
encoded
point cloud
Prior art date
Application number
PCT/CN2020/080140
Other languages
English (en)
Chinese (zh)
Inventor
蔡康颖
张德军
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201910214557.8A external-priority patent/CN111726616B/zh
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20774250.3A priority Critical patent/EP3934250A4/fr
Priority to SG11202110220RA priority patent/SG11202110220RA/en
Priority to BR112021018435A priority patent/BR112021018435A2/pt
Priority to JP2021556407A priority patent/JP7453992B2/ja
Priority to KR1020217033214A priority patent/KR20210134401A/ko
Publication of WO2020187283A1 publication Critical patent/WO2020187283A1/fr
Priority to US17/477,812 priority patent/US20220007037A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties

Definitions

  • This application relates to the field of data processing technology, and in particular to a point cloud encoding method, point cloud decoding method, device and storage medium.
  • This application provides a point cloud encoding method, point cloud decoding method, device and storage medium, which can solve the problem of how to effectively encode and decode point clouds in related technologies.
  • the technical solution is as follows:
  • a point cloud coding method comprising: obtaining auxiliary information of a patch to be coded (point cloud block); encoding the auxiliary information of the patch to be coded and a first identifier of the patch to be coded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a patch of the first type, and the auxiliary information of the patch to be encoded includes three-dimensional information;
  • the first identifier is a second value to indicate that the patch to be encoded is a patch of the second type,
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is the third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded Including two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream. Moreover, for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction. Coordinate axis information.
  • the first identifier is the second value, it can indicate that the patch to be encoded is the second type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, and Does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis between the first-type patches in the current point cloud group
  • the information can be the same, but the three-dimensional information can be different.
  • the second type of patch has a matching relationship in the reference frame of the current point cloud frame, and the rotation transformation information and normal coordinate axis between the second type of patch and its reference patch The information can be the same, while the two-dimensional information and the three-dimensional information can be different.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the bit stream, and then improve the coding Efficiency.
  • the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first type of patch can be a global matching patch in the current group
  • the second type of patch can be a local matching patch in the current group
  • the third type of patch can be a non-matching patch in the current group.
  • the first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively.
  • the reference patch of the patch to be encoded may be included in the reference frame of the current point cloud frame.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream.
  • the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream.
  • the first identification of patch[i-1] to be encoded and the auxiliary information of patch[i-1] to be encoded, the first identification of patch[i] to be encoded and patch[ i] auxiliary information, as well as the first identifier of the patch[i+1] to be coded and the auxiliary information of the patch[i+1] to be coded are sequentially encoded into the bitstream, that is, the current point cloud frame
  • the first identifier of each patch is individually packaged and compiled into the code stream.
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. Use fixed-length encoding to encode the first identifier of the patch to be encoded into the code stream.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded.
  • the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method.
  • the specific encoding method is not limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the embodiment of the present application may divide the current point cloud group
  • the first identifier of all patches included in the point cloud of the first frame is set to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value.
  • the first identifier of the first patch in the current point cloud frame is the first value.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame
  • it can be determined that the current point cloud frame is the first point cloud in the current point cloud group .
  • the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch.
  • the patch to be coded is the first type 3 patch, then it can be coded according to the fixed-length encoding method. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the first type 3 patch
  • the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded using the fixed-length encoding method described above.
  • the auxiliary information of the patch to be encoded may also include identification information of the reference patch of the patch to be encoded.
  • the identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded.
  • the identification information of the reference patch may be the position number of the reference patch in the reference frame.
  • a point cloud decoding method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the auxiliary information of the patch to be decoded from the code stream Information, the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded
  • the patch refers to the identification information of the patch; when the first identification is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information: reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • the first identifier when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is of the second type, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third-type patch, that is, there is no reference patch that matches the patch to be decoded in the reference frame of the current point cloud frame.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • a method that matches the encoding end can be used for decoding, for example, a patch whose position in the decoded point cloud is the same as that of the patch to be decoded in the current point cloud frame Use differential decoding to analyze the 3D information of the patch to be decoded from the code stream.
  • auxiliary information of the patch to be decoded for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information
  • the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information.
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained
  • the axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. That is, for the first type of patch, three-dimensional information can be parsed from the code stream, while the two-dimensional information, rotation transformation information, and normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream Two-dimensional information, rotation transformation information and normal coordinate axis information also reduce the bit overhead of the code stream and improve coding efficiency at the same time.
  • the auxiliary information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the to-be-decoded patch to assist
  • the information includes two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the decoded patch.
  • the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded A patch with the same position in the current point cloud frame.
  • the first type of patch, the second type of patch and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in sequence.
  • the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded.
  • the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. That is, for the second type of patch, two-dimensional information and three-dimensional information can be parsed from the code stream, while the rotation transformation information and the normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream
  • the rotation transformation information and the normal coordinate axis information also reduce the bit overhead of the code stream and improve the coding efficiency at the same time.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  • the first identifier is the third value
  • a method that matches the encoding end can be used for decoding, for example, fixed-length decoding is used to parse the two-dimensional information, three-dimensional information, and rotation of the patch to be decoded from the code stream. Transformation information and normal axis information.
  • the auxiliary information of the first third-type patch is coded according to the fixed-length coding method
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by the differential coding method
  • other auxiliary information is coded by the fixed-length coding method. coding.
  • the two-dimensional information and three-dimensional information of the patch to be decoded are parsed from the code stream by differential decoding, and fixed
  • the long decoding method parses the rotation transformation information and normal coordinate axis information of the patch to be decoded from the code stream.
  • the non-first third type of patch two-dimensional information and three-dimensional information can be parsed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so as to avoid
  • the bit overhead of the code stream is also reduced, and the coding efficiency is improved at the same time.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the first type patch, the second type patch and the third type patch in each frame of point cloud in the current point cloud group can be arranged in sequence.
  • the patch to be decoded is the first one in the current point cloud frame patch
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in order, that is, any of the current point cloud group
  • the first identifier of the first patch in a frame of point cloud can be the first value, the second value or the third value.
  • the first identifiers of multiple patches included in the current point cloud frame can be determined.
  • the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • a point cloud coding method includes: obtaining auxiliary information of the patch to be coded; encoding the auxiliary information of the patch to be coded and the first identifier of the patch to be coded into a code stream; wherein the first identifier is The first value indicates that the patch to be encoded is a first-type patch, and the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream ; Or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, and the auxiliary information of the patch to be encoded includes three-dimensional information; wherein the first identifier is the second value to indicate that the patch to be encoded is For the second type of patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the
  • the first identifier When the first identifier is the first value, it may indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream may also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identification may be override_3d_shift_data_flag.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value
  • the syntax element of the fourth value may be true, and the syntax element of the fifth value may be false.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • a point cloud decoding method includes: parsing the first identifier of the patch to be decoded from the code stream; when the first identifier is the first value, parsing the first identifier of the patch to be decoded from the code stream Second identifier; when the second identifier is the fourth value, the auxiliary information of the reference patch of the patch to be decoded is used as the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the patch to be decoded is parsed from the code stream Auxiliary information.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, and the patch to be decoded.
  • the identification information of the reference patch of the decoded patch; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and method of the patch to be decoded. To coordinate axis information; according to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded.
  • the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream.
  • a point cloud encoding device includes: a patch information acquisition module for auxiliary information of the patch to be encoded; an auxiliary information encoding module for combining auxiliary information of the patch to be encoded and the auxiliary information of the patch to be encoded
  • the first identifier is encoded into the code stream; the first identifier is the first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information; the first identifier is the second value to indicate the The coded patch is a second type of patch, and the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be coded; the first identifier is the third value to indicate that the patch to be coded is the third type Patch, the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • a point cloud decoding device includes: an auxiliary information decoding module for parsing the first identifier of the patch to be decoded from the code stream; the auxiliary information decoding module is also used for when the first identifier is the first identifier When the value is one, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, auxiliary information Including the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the patch to be decoded
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the auxiliary information decoding module is also used for:
  • the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  • the auxiliary information decoding module is also used for:
  • the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
  • the auxiliary information decoding module is further configured to: obtain the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;
  • the reconstruction module is also used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
  • the auxiliary information decoding module is also used to: obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; Used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • a point cloud encoding device including: a patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
  • Auxiliary information encoding module used to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first-type patch
  • the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded
  • the code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • a point cloud decoding device including: an auxiliary information decoding module, configured to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module is also used for parsing the second identification of the patch to be decoded from the code stream when the first identification is the first value; when the second identification is the fourth value, referencing the auxiliary information of the patch to be decoded As the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • a point cloud encoding method comprising: obtaining auxiliary information of the patch to be encoded; encoding the auxiliary information of the patch to be encoded and the syntax elements of the patch to be encoded into a code stream, and the syntax elements include the first A grammatical element.
  • the syntax element encoded into the code stream also includes a second syntax element.
  • the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not In the code stream, the auxiliary information of the patch to be coded includes the two-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the three-dimensional information of the patch to be coded is coded into the code stream, and the auxiliary information of the patch to be coded includes the three-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not Code stream, and the 3D information of the patch to be coded is not coded into code stream.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first syntax element indicates that the patch to be encoded does not have a reference patch
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
  • a point cloud decoding method includes: parsing a first syntax element of a patch to be decoded from a code stream; when the first syntax element indicates that the patch to be decoded has a reference patch, read from the code stream Parse the second syntax element of the patch to be decoded, or parse the second syntax element and the third syntax element of the patch to be decoded from the code stream; when the value of the second syntax element is true, from the code Parse the two-dimensional information of the patch to be decoded in the stream, and use the three-dimensional information of the reference patch of the patch to be decoded as the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false When the value is true, parse the three-dimensional information of the patch to be decoded from the code stream, and use the two-dimensional information of the reference patch of the patch to be decoded as the two-dimensional information of the code stream
  • the method further includes: when the first syntax element indicates that the patch to be decoded does not have a reference patch, parsing auxiliary information of the patch to be decoded from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information, Three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the method further includes: when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch, determining that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame .
  • parsing the first syntax element of the patch to be decoded from the code stream includes: when the patch to be decoded is the first patch in the current point cloud frame, parsing the first syntax element of the patch to be decoded from the code stream Element; or, when the patch to be decoded is the first patch in the current point cloud frame, parse multiple first syntax elements of multiple patches included in the current point cloud frame from the code stream, and multiple parsed ones
  • the first syntax element includes the first syntax element of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
  • a point cloud encoding device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the patch to be encoded;
  • the auxiliary information encoding module is used to encode the auxiliary information of the patch to be encoded and the syntax element of the patch to be encoded into the code stream, and the syntax element includes the first syntax element, wherein:
  • the syntax element encoded into the code stream also includes a second syntax element.
  • the value of the second syntax element is true to indicate that the two-dimensional information of the patch to be encoded is incorporated into the code stream, but the three-dimensional information of the patch to be encoded is not encoded.
  • the auxiliary information of the patch to be encoded includes two-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is true to indicate that the two-dimensional information of the patch to be encoded is not
  • the code stream is encoded, and the three-dimensional information of the patch to be encoded is encoded into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information; or
  • the syntax element encoded into the code stream also includes a second syntax element and a third syntax element.
  • the value of the second syntax element is false and the value of the third syntax element is false to indicate that the two-dimensional information of the patch to be encoded is not
  • the code stream is compiled, and the three-dimensional information of the patch to be coded is not compiled into the code stream.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first syntax element indicates that the patch to be encoded does not have a reference patch
  • the patch to be encoded is the first patch in the current point cloud frame to which it belongs
  • the current point cloud frame is the first point cloud frame
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be encoded in the current point cloud frame is the same as the position of the reference patch of the patch to be encoded in the reference frame of the current point cloud frame.
  • a point cloud decoding device includes: an auxiliary information decoding module, configured to parse the first syntax element of the patch to be decoded from the code stream;
  • the auxiliary information decoding module is also used to parse the second syntax element of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded has a reference patch, or to parse the code stream to be decoded
  • the second syntax element and the third syntax element of the patch when the value of the second syntax element is true, the two-dimensional information of the patch to be decoded is parsed from the code stream, and the three-dimensional information of the patch to be decoded is referenced As the three-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is true, the three-dimensional information of the patch to be decoded is parsed from the code stream, and the The two-dimensional information of the reference patch of the decoded patch is used as the two-dimensional information of the patch to be decoded; or, when the value of the second syntax element is false and the value of the third syntax element is false, the reference of the patch to be decoded The two
  • the reconstruction module is used for reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes the two-dimensional information of the patch to be decoded and the three-dimensional information of the patch to be decoded.
  • the auxiliary information decoding module is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first syntax element indicates that the patch to be decoded does not have a reference patch.
  • the auxiliary information of the patch to be decoded includes two Dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the auxiliary information decoding module is further configured to determine that the current point cloud frame to which the patch to be decoded belongs is the first point when the patch to be decoded is the first patch and the first syntax element indicates that the patch to be decoded does not have a reference patch Cloud frame.
  • the auxiliary information decoding module is further configured to parse the first syntax element of the patch to be decoded from the code stream when the patch to be decoded is the first patch in the current point cloud frame; or, when the patch to be decoded is the first patch in the current point cloud frame;
  • the decoded patch is the first patch in the current point cloud frame
  • the multiple first syntax elements of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first syntax elements parsed include the The first syntax element of the patch to be decoded.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the position of the patch to be decoded in the current point cloud frame is the same as the position of the reference patch of the patch to be decoded in the reference frame of the current point cloud frame.
  • a device for encoding point cloud data may include a memory and an encoder.
  • the memory is used to store point cloud data.
  • the encoder is used to implement the point cloud coding method provided by the first aspect or any possible design of the first aspect, or the encoder is used to implement the points provided by the third aspect or any possible design of the third aspect.
  • the cloud coding method, or the encoder is used to execute the point cloud coding method provided by any one of the possible designs of the ninth aspect or the ninth aspect.
  • a device for decoding point cloud data may include a memory and a decoder.
  • the memory is used to store point cloud data in the form of code stream.
  • the decoder is used to implement the point cloud decoding method provided by the foregoing second aspect or any possible design of the second aspect, or the decoder is used to implement the points provided by any possible design of the foregoing fourth aspect or the fourth aspect
  • the cloud decoding method, or the decoder is used to execute the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
  • an encoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the first aspect or the third aspect or the ninth aspect.
  • Point cloud coding method provided by the aspect.
  • a decoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the second aspect or the fourth aspect or the tenth aspect.
  • the point cloud decoding method provided by the aspect.
  • This application also provides a computer-readable storage medium, including program code, when the program code runs on a computer, the computer executes the first aspect and its possible designs, or the third aspect and its possible designs. Any point cloud coding method in the ninth aspect and its possible designs.
  • This application also provides a computer-readable storage medium, including program code, which when running on a computer, causes the computer to execute the second aspect and its possible designs, or the fourth aspect and its possible designs. Any point cloud decoding method, or any point cloud decoding method provided by the tenth aspect and its possible designs.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the first aspect or the first aspect, or execute the third aspect or the third aspect Part or all of the steps of the point cloud coding method provided by any possible design, or execute part or all of the steps of the point cloud coding method provided by any possible design of the ninth aspect or the ninth aspect.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the second aspect or the second aspect, or execute the fourth aspect or the fourth aspect Part or all of the steps of the point cloud decoding method provided by any possible design, or execute part or all of the steps of the point cloud decoding method provided by any possible design of the tenth aspect or the tenth aspect.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application;
  • Fig. 2 is a schematic block diagram of an encoder that can be used in an embodiment of the present application
  • Fig. 3 is a schematic diagram of a point cloud applicable to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a point cloud patch applicable to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application.
  • Fig. 6 is a schematic block diagram of a decoder that can be used in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a frame group that can be used in an embodiment of the present application.
  • FIG. 8 is a schematic diagram of the first current point cloud group provided by an embodiment of this application.
  • FIG. 9 is a flowchart of the first point cloud encoding method provided by an embodiment of the application.
  • FIG. 10 is a schematic diagram of a second current point cloud group provided by an embodiment of this application.
  • FIG. 11 is a flowchart of the first point cloud decoding method provided by an embodiment of this application.
  • FIG. 12 is a flowchart of a second point cloud encoding method provided by an embodiment of this application.
  • FIG. 13 is a flowchart of a second point cloud decoding method provided by an embodiment of this application.
  • FIG. 14 is a schematic block diagram of a first encoding device provided by an embodiment of this application.
  • FIG. 15 is a schematic block diagram of a first decoding device provided by an embodiment of this application.
  • FIG. 16 is a schematic block diagram of a second encoding device provided by an embodiment of this application.
  • FIG. 17 is a schematic block diagram of a second decoding device provided by an embodiment of this application.
  • FIG. 18 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application.
  • the term "point cloud decoding” or “decoding” may generally refer to point cloud encoding or point cloud decoding.
  • the point cloud decoding system includes a source device 10, a destination device 20, a link 30 and a storage device 40.
  • the source device 10 can generate coded point cloud data. Therefore, the source device 10 may also be referred to as a point cloud encoding device.
  • the destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device.
  • the link 30 can receive the coded point cloud data generated by the source device 10 and can transmit the coded point cloud data to the destination device 20.
  • the storage device 40 can receive the coded point cloud data generated by the source device 10, and can store the coded point cloud data, so that the destination device 20 can directly obtain the coded point cloud from the storage device 40 data.
  • the storage device 40 may correspond to a file server or another intermediate storage device that can save the encoded point cloud data generated by the source device 10, so that the destination device 20 may stream or download the stored data of the storage device 40 Coded point cloud data.
  • Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors.
  • the memory may include random access memory (RAM), read-only memory ( read-only memory, ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, which can be used to store the desired program in the form of instructions or data structures that can be accessed by a computer Any other media of the code, etc.
  • both the source device 10 and the destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.
  • the link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20.
  • the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded point cloud data to the destination device 20 in real time.
  • the source device 10 may modulate the coded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may send the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and/or wired communication media.
  • the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, and the packet-based network may be a local area network, a wide area network, or a global network (for example, the Internet).
  • the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application does not specifically limit this.
  • the storage device 40 may store the received encoded point cloud data sent by the source device 10, and the destination device 20 may directly obtain the encoded point cloud data from the storage device 40 .
  • the storage device 40 may include any of a variety of distributed or locally accessed data storage media.
  • any of the multiple distributed or locally accessed data storage media may be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or used to store Any other suitable digital storage media for encoding point cloud data, etc.
  • the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, and the destination device 20 can be stored via streaming or downloading.
  • the file server may be any type of server capable of storing the coded point cloud data and transmitting the coded point cloud data to the destination device 20.
  • the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive.
  • FTP file transfer protocol
  • NAS network attached storage
  • the destination device 20 can obtain the coded point cloud data through any standard data connection (including an Internet connection).
  • Any standard data connection can include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, digital subscriber line (DSL), cable modem, etc.), or is suitable for obtaining the encoded data stored on the file server The combination of the two point cloud data.
  • the transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.
  • the point cloud decoding system shown in FIG. 1 is only one possible implementation, and the technology of the present application can not only be applied to the source device 10 that can encode the point cloud shown in FIG.
  • the destination device 20 for decoding point cloud data may also be applicable to other devices that can encode point clouds and decode encoded point cloud data, which is not specifically limited in the embodiment of the present application.
  • the source device 10 includes a data source 120, an encoder 100 and an output interface 140.
  • the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, where the transmitter may also be referred to as a transmitter.
  • the data source 120 may include a point cloud capture device (for example, a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or A computer graphics system used to generate point cloud data, or a combination of these sources of point cloud data.
  • the data source may send a point cloud to the encoder 100, and the encoder 100 may encode the point cloud received from the data source 120 to obtain encoded point cloud data.
  • the encoder can send the encoded point cloud data to the output interface.
  • the source device 10 directly sends the encoded point cloud data to the destination device 20 via the output interface 140.
  • the encoded point cloud data may also be stored on the storage device 40 for the destination device 20 to obtain later and use for decoding and/or playback.
  • the destination device 20 includes an input interface 240, a decoder 200 and a display device 220.
  • the input interface 240 includes a receiver and/or a modem.
  • the input interface 240 can receive the encoded point cloud data via the link 30 and/or from the storage device 40, and then send it to the decoder 200.
  • the decoder 200 can decode the received encoded point cloud data to obtain the Decoded point cloud data.
  • the decoder may send the decoded point cloud data to the display device 220.
  • the display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data.
  • the display device 220 may be any one of multiple types of display devices.
  • the display device 220 may be a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED). Display or other type of display device.
  • LCD liquid crystal display
  • OLED organic light-emitting
  • the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software, used for encoding both audio and video in a common data stream or separate data streams.
  • MUX-DEMUX multiplexer-demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
  • Each of the encoder 100 and the decoder 200 can be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuit, ASIC ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors.
  • Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or the decoders may be integrated as a combined encoding in the corresponding device Part of the decoder/decoder (codec).
  • This application may generally refer to the encoder 100 as “signaling” or “sending” certain information to another device such as the decoder 200.
  • the term “signaling” or “sending” may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. retrieve the syntax element at any time.
  • FIG. 2 is a schematic block diagram of an encoder 100 provided by an embodiment of the application.
  • Fig. 2 illustrates the point cloud compression (PCC) coding framework of MPEG (moving pictrue expert group) as an example.
  • the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, image-based or video-based Encoding module 107, occupancy map encoding module 108, auxiliary information encoding module 109, multiplexing module 110, point cloud occupancy map down-sampling module 111, point cloud occupancy map filling module 112, point cloud reconstruction module 113, and point cloud filtering module 114 .
  • PCC point cloud compression
  • the point cloud block information generation module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds.
  • each point cloud group may include one or more frames of point clouds.
  • the current point cloud group will be unified in the following Take the current point cloud frame in as an example.
  • the point cloud block information generation module 101 can determine the three-dimensional coordinates of each point included in the current point cloud frame in the three-dimensional space coordinate system, and the normal direction vector of each point in the three-dimensional space, and determine each point according to the determined
  • the normal direction vector in the three-dimensional space and the predefined projection plane divide the current point cloud frame into multiple patches (patch). Each patch includes one or more points in the current point cloud frame.
  • the point cloud block information generating module 101 can also determine the occupancy map of each patch and the depth map of each patch.
  • the point cloud block information generating module 101 can also determine the three-dimensional information of each patch and the normal coordinate axis information of each patch, which can be used as auxiliary information of each patch.
  • the point cloud block information generating module 101 can divide the patch into three types, namely the first type patch, the second type patch, and the third type patch.
  • the first identifier is used to indicate the type of each patch.
  • the point cloud block information generating module 101 may send the first identification of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be called compression encoding.
  • the three-dimensional information of each patch may include the offset of the patch to be encoded in the 3D space along the tangent direction, the offset of the patch to be encoded in the 3D space along the double tangent direction, and the offset of the patch to be encoded in the 3D space. Offset in the normal direction.
  • the point cloud block information generating module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102.
  • the point cloud block information generating module 101 can also send the depth map of each patch to the depth map generating module 103.
  • the packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101 to obtain the occupancy map of the current point cloud frame.
  • the packaging module 102 can arrange the occupancy map of each patch in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch, and then arrange the occupancy map of each patch in descending order (or ascending order). In the order of the occupancy map, insert the occupancy map of each patch into the available area of the occupancy map of the current point cloud frame to obtain the occupancy map of the current point cloud frame and the packing information of the patch.
  • the packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding.
  • the packing information of the patch may include the two-dimensional information of each patch in the occupancy map of the current point cloud frame, the identification information of the reference patch of each patch, the rotation transformation information of each patch, and the reference of the current point cloud frame The index of the frame.
  • the two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch.
  • the two-dimensional information of each patch can include the offset of each patch along the u axis in the current point cloud frame occupancy map, the offset of each patch along the v axis in the current point cloud frame occupancy map, and each patch
  • the current point cloud frame occupies the width of the figure, and each patch occupies the height of the figure in the current point cloud frame.
  • the packing module 102 may also send the packing information of the patch to the depth map generating module 103 and the point cloud occupancy map down-sampling module 111.
  • FIG. 4 is a schematic diagram of the patch of the point cloud of the frame
  • Fig. 5 is the occupancy diagram of each patch of the point cloud of the frame shown in Fig. 4, the frame point obtained by packing by the packing module 102 Schematic diagram of cloud occupancy map.
  • the point cloud shown in FIG. 3 may be the current point cloud frame in the embodiment of the present application
  • the patch of the point cloud shown in FIG. 4 may be the patch of the current point cloud frame in the embodiment of the present application
  • the cloud occupancy map may be the occupancy map of the current point cloud frame in the embodiment of the present application.
  • the auxiliary information encoding module 109 receives the occupancy map of the current point cloud frame and the packing information of the patch sent by the packing module 102, as well as the first identification of each patch and the first identification of each patch sent by the point cloud block information generating module 101. After the auxiliary information, the current point cloud frame and each patch included in the current point cloud frame can be encoded to obtain a code stream including the encoded auxiliary information, and then the obtained code stream including the encoded auxiliary information is sent to the complex Use module 110. Among them, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the packaging information of the patch may also be referred to as auxiliary information of each patch.
  • the depth map generation module 103 may receive the patch packaging information sent by the packaging module 102 and the depth map of each patch sent by the point cloud block information generation module 101, and then according to the packaging information of each patch and each The depth map of the patch generates the depth map of the current point cloud frame. Then the generated depth map of the current point cloud frame is sent to the depth map filling module 105 to fill the blank pixels in the depth map of the current point cloud frame to obtain the filled depth map of the current point cloud frame.
  • the depth map filling module 105 may send the obtained filled depth map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled depth map of the current point cloud frame, Obtain the reconstructed current point cloud depth map and the code stream including the encoded current point cloud frame depth map, and may send the obtained reconstructed current point cloud frame depth map to the point cloud reconstruction module 113 , And send the code stream including the encoded depth map of the current point cloud frame to the multiplexing module 110.
  • the point cloud occupancy map down-sampling module 111 may perform down-sampling processing on the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain a low-resolution occupancy map of the current point cloud frame. After that, the point cloud occupancy map down-sampling module 111 may also send the occupancy map of the current low-resolution point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112.
  • the occupancy map encoding module 108 can encode the received occupancy map of the current low-resolution point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 can also The code stream including the coded low-resolution occupancy map of the current point cloud frame may be sent to the multiplexing module 110.
  • the point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the low-resolution current point cloud frame to obtain the filled occupancy map of the current point cloud frame, The occupancy map of the filled current point cloud frame has the original resolution.
  • the point cloud reconstruction module 113 may be based on the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, and the reconstructed current point cloud frame sent by the image or video-based encoding module 107.
  • the depth map, patch packing information and patch auxiliary information reconstruct the geometry of the current point cloud frame to output the reconstructed point cloud.
  • the point cloud reconstruction module 113 can also output the reconstructed points in the reconstructed point cloud. Correspondence with patch, and the packaging position of the reconstructed point in the reconstructed point cloud.
  • the point cloud reconstruction module 113 can send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 can also reconstruct the reconstructed point cloud.
  • the packing positions of the points are sent to the texture generation module 104.
  • the point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be referred to as a smooth reconstructed point cloud. Or it can be said that the point cloud filtering block 114 can perform smoothing processing on the reconstructed point cloud.
  • the texture map generating module 104 receives the smooth reconstructed point cloud sent by the point cloud filtering module 114, the packed position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstruction module 113, and is sent by the data source 120 After the current point cloud frame, the texture map of the current point cloud frame can be generated according to the smooth reconstructed point cloud, the packaging position of the reconstructed point in the reconstructed point cloud and the current point cloud frame, and the generated current point cloud The texture map of the frame is sent to the texture map filling module 106 to fill the blank pixels in the texture map of the current point cloud frame to obtain the filled texture map of the current point cloud frame.
  • the texture map filling module 106 may send the obtained filled texture map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled texture map of the current point cloud frame, Obtain the code stream including the texture map of the reconstructed current point cloud frame.
  • the image or video-based encoding module 107 may also send the obtained code stream including the reconstructed texture map of the current point cloud frame to the multiplexing module 110.
  • the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 can send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 can combine the received code streams To form a combined code stream, and send the combined code stream to the output interface 140.
  • the output interface 140 may send the combined code stream to the decoder 200.
  • the encoder 100 shown in FIG. 2 is only an embodiment provided by the present application.
  • the encoder 100 may include more or less modules than those shown in FIG. 2 Module.
  • the embodiments of the present application do not specifically limit this.
  • FIG. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the application.
  • Figure 6 illustrates the MPEG PCC decoding framework as an example.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, and a point cloud reconstruction module 206 , Point cloud filtering module 207 and point cloud texture information reconstruction module 208.
  • the demultiplexing module 201 may receive the combined code stream sent by the output interface 140 of the encoder 100 through the input interface 204, and send the combined code stream to the corresponding decoding module. Specifically, the demultiplexing module 201 sends the code stream of the texture map of the current point cloud frame encoded and the code stream of the depth map of the encoded current point cloud frame to the image or video-based decoding module 202, and The code stream including the occupancy map of the current point cloud frame with the encoded low resolution is sent to the occupancy map decoding module 203, and the code stream including the encoded auxiliary information is sent to the auxiliary information decoding module 204.
  • the image or video-based decoding module 202 can decode the received bitstream including the texture map of the encoded current point cloud frame and the bitstream including the depth map of the encoded current point cloud frame to obtain the reconstructed current
  • the texture map information of the point cloud frame and the depth map information of the reconstructed current point cloud frame, and the reconstructed texture map information of the current point cloud frame can be sent to the point cloud texture information reconstruction module 208,
  • the constructed depth map information of the current point cloud frame is sent to the point cloud reconstruction module 206.
  • the occupancy map decoding module 203 can decode the received bitstream including the occupancy map of the current point cloud frame with encoded low resolution to obtain the occupancy map information of the reconstructed current point cloud frame with low resolution, and
  • the reconstructed low-resolution current point cloud frame occupancy map information is sent to the point cloud occupancy map filling module 205.
  • the point cloud occupancy map filling module 205 can obtain the reconstructed occupancy map information of the current point cloud frame with the original resolution according to the occupancy map information of the reconstructed low-resolution current point cloud frame, and then the reconstructed has The occupancy map information of the current point cloud frame at the original resolution is sent to the point cloud reconstruction module 206.
  • the reconstructed occupancy map information of the current point cloud frame with the original resolution will be uniformly referred to as the occupancy map information of the reconstructed current point cloud frame.
  • the auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the point cloud reconstruction module 206.
  • the specific decoding method will be described in subsequent embodiments.
  • the point cloud reconstruction module 206 can be based on the received depth map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, and the reconstructed current point cloud sent by the point cloud occupancy map filling module 205.
  • the occupancy map information of the frame and the auxiliary information sent by the auxiliary information decoding module 204 reconstruct the geometry of the current point cloud frame to obtain the reconstructed point cloud.
  • the reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process can refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100. I won't repeat it here.
  • the point cloud reconstruction module 206 may also send the reconstructed point cloud to the point cloud filtering module 207.
  • the point cloud filtering module 207 can filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud.
  • the specific filtering process can refer to the filtering process of the point cloud filtering module 114 in the encoder 100 , I won’t repeat it here.
  • the point cloud filtering module 207 may send the smooth reconstructed point cloud to the texture information reconstruction module 208 of the point cloud.
  • the texture information reconstruction module 208 of the point cloud receives the smooth reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202. Afterwards, the texture information of the reconstructed point cloud can be reconstructed to obtain the reconstructed point cloud reconstructed by the texture information.
  • the decoder 200 shown in FIG. 6 is only an example, and in a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 6. The embodiment of the present application does not limit this.
  • the current point cloud group may be a group of frame (GOF), and a frame group may include 32 frames of point clouds, that is, the current point cloud group may include 32 frames of point clouds.
  • the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application.
  • a frame group may also be divided into multiple subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the multiple subgroups.
  • FIG. 7 is a schematic diagram of a frame group provided by an embodiment of the application. As shown in FIG.
  • a frame group can be divided into 10 subgroups, and the first subgroup includes the first, second, and third subgroups.
  • Frame point cloud the second subgroup includes the 4th frame point cloud
  • the third subgroup includes the 5th, 6, 7, and 8th frame point clouds
  • the 10th subgroup includes the 30th, 31st, and 32th frame point clouds
  • the current point cloud group can be any one of these 10 subgroups.
  • FIG. 7 only exemplarily shows the point clouds included in the first subgroup, the second subgroup, the third subgroup, and the tenth subgroup, and the point clouds included in other subgroups are not described in detail.
  • the current point cloud group can be a frame group, or the current point cloud group can be a subgroup of a frame group.
  • the current point cloud group can also be other forms of point cloud groups, such as custom ones.
  • the point cloud group is not limited in this embodiment of the application.
  • patches can be divided into three types, namely the first type of patch, the second type of patch and the third type of patch.
  • the patch can also be divided into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, the first type of patch can be a global matching patch, the second type of patch can be a local matching patch, and the third type of patch can be a non-matching patch.
  • the three types of patches are described.
  • the patch to be coded is any patch in the current point cloud frame, and the current point cloud frame is any of the current point cloud groups.
  • a frame of point cloud If the patch to be coded is the first type of patch, the patch to be coded has a matching relationship in all the point cloud frames in the current point cloud group, this can be called a global matching patch; if the patch to be coded is the second For the patch type, the patch to be encoded has a patch with a matching relationship in the reference frame of the current point cloud frame, but only a patch with a matching relationship exists in a part of the point cloud frame in the current point cloud group.
  • the reference frame of the current point cloud frame may be located in the current point cloud group.
  • the reference frame of the current point cloud frame may be the point cloud of the previous frame of the current point cloud frame.
  • the patches included in all point clouds in the current point cloud group can be arranged in the order of the first type patch, the second type patch, and the third type patch.
  • the first type patch in the current point cloud group The number of point clouds in the frame is the same, and when the number of the first-type patches is greater than 1, the first-type patches in each frame of the current point cloud group can also be arranged in the same order, that is, Yes, there is a matching relationship between the first-type patches at the same position in the point cloud of each frame.
  • the w-th first-type patch in each frame of point cloud has a matching relationship with each other, and w is less than or equal to the number of first-type patches.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the first-type patch in the occupancy map of the current point cloud frame are the same, but the three-dimensional information is different.
  • the two-dimensional information of the first-type patch can also be different, or the three-dimensional information can also be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. In this way, for two-dimensional information and three-dimensional information, there may be three situations.
  • the following takes the patch to be coded as an example.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded Same as the three-dimensional information of the patch to be encoded.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded
  • the three-dimensional information of the reference patch of the patch to be encoded is the same as The three-dimensional information of the patch to be encoded is different.
  • the two-dimensional information of the reference patch of the patch to be encoded is different from the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded The information is different from the three-dimensional information of the patch to be encoded.
  • the reference patch of the patch to be decoded may be the same position in the reference frame of the current point cloud frame where the patch to be decoded is located. patch. For example, if the patch to be decoded is in the third position in the current point cloud frame, then the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be decoded.
  • the rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information.
  • the rotation angle information includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch to be coded is rotated by 20°; if the rotation angle of the patch to be coded is A value of 80 means that the patch to be coded is rotated by 80°.
  • the rotation angle index of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch to be coded is rotated by 15°; if the rotation angle of the patch to be coded is If the index is 2, it means that the patch to be coded is rotated by 30°; if the rotation angle index of the patch to be coded is 3, it means that the patch to be coded is rotated by 45°.
  • the rotation angle information can also be characterized in other forms, which is not specifically limited here. For example, it can also be described in the form of a matrix, especially when acceleration processing is performed in a GPU (Graphic Processing Unit, image processor), the use of the matrix form will increase the processing speed.
  • rotation of the patch to be encoded can be clockwise or counterclockwise.
  • the coordinate axis exchange information can be expressed as the coordinate axis exchange flag.
  • the coordinate axis exchange flag when the coordinate axis exchange flag is false or 0, it indicates the tangent coordinate axis of the patch to be encoded and the double tangent coordinate of the patch to be encoded
  • the axes are not exchanged; for example, when the coordinate axis exchange flag is ture or 1, the tangent axis of the patch to be encoded and the double tangent axis of the patch to be encoded are exchanged.
  • the index of the coordinate axis of the patch to be encoded can be 0, 1, or 2.
  • the tangent axis index of the patch to be encoded and the double tangent axis index of the patch to be encoded can be 0, 1, and 2. Any two.
  • the tangent axis of the patch can be called the U axis
  • the double tangent axis of the patch can be called the V axis.
  • the obtained tangent axis index of the current patch and the double tangent axis index of the patch to be encoded are 0 and 2, respectively.
  • the tangent of the patch to be encoded is not exchanged
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100.
  • FIG. 9 is a flowchart of a point cloud encoding method provided by an embodiment of the present application, and the method is applied to a point cloud decoding system. Referring to Figure 9, the method includes:
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • S902 Encode the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc.
  • the value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch.
  • the auxiliary information of the patch to be encoded encoded into the code stream includes three-dimensional information, but does not include two-dimensional information, rotation transformation information, and normal direction.
  • Coordinate axis information when the first identifier is the second value, it can indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded that is encoded into the code stream includes two-dimensional information, three-dimensional information, and the patch to be encoded
  • the identification information of the reference patch does not include the rotation transformation information and the normal coordinate axis information; and/or, when the first identification is the third value, it can indicate that the patch to be encoded is a third-type patch, and is encoded into the waiting stream of the code stream.
  • the auxiliary information of the encoded patch includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the first-type patch has matching patches in all point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information and method between the first-type patches in the current point cloud group
  • the direction axis information is the same, but the three-dimensional information may be different.
  • the second type of patch has a matching patch in the reference frame of the current point cloud frame, and the rotation transformation information and normal direction between the second type of patch and its reference patch
  • the coordinate axis information is the same, but the two-dimensional information and the three-dimensional information may not be the same.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited.
  • the fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be encoded into the code stream according to the two-dimensional information and three-dimensional information of the reference patch to be encoded.
  • the differential encoding method can be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the two-dimensional information and three-dimensional information of the encoded patch are encoded into the code stream, and the two-dimensional information and the three-dimensional information of the patch to be encoded can also be encoded into the code stream by using a more accurate predictive difference method.
  • the specific encoding method is not limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and the first identifier of the patch to be encoded into the code stream.
  • encoding the three-dimensional information of the patch to be encoded into the code stream using the differential encoding method refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be decoded into the code stream.
  • Using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be decoded, and the patch to be encoded
  • the difference between the 3D information of the reference patch and the 3D information of the patch to be decoded is encoded into the code stream.
  • the first identifiers of multiple patches in the current point cloud frame can be packaged together and compiled into a code stream.
  • the first identifiers of all patches in the current point cloud frame can be packaged together and compiled into the code stream.
  • the first identifier of each patch in the current point cloud frame can be individually packaged and compiled into the code stream.
  • the first identifier of the patch[i-1] to be encoded can be combined with The auxiliary information of the patch[i-1] to be encoded, the first identification of the patch[i] to be encoded and the auxiliary information of the patch[i] to be encoded, and the first identification of the patch[i+1] to be encoded and the patch to be encoded [i+1] Auxiliary information, etc. are sequentially encoded into the code stream. That is, for the current point cloud frame, the first identifiers of all patches in the current point cloud frame can be packaged separately and compiled into a code stream, or they can be packed together and compiled into a code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, as shown in Table 1 and Table 2.
  • skip_patch_data_unit in Table 1 is the information of the patch to be encoded that is encoded into the code stream when the first identifier of the patch to be encoded is the first value, and can also be referred to as the data unit of the patch to be encoded.
  • the embodiment of the present application may After receiving the first frame point cloud of the current point cloud group, the auxiliary information encoding module 109 sets the first identifiers of all patches included in the first frame point cloud of the current point cloud group to the third value. In this way, when the current point cloud frame is the first point cloud in the current point cloud group, the first identifier of the first patch in the current point cloud frame is the third value.
  • the first identifier of the first patch in the current point cloud frame is the first value. That is, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group . In this way, there is no need to include the length of the point cloud frame of the current point cloud group in the code stream, thereby improving the coding efficiency and reducing the bit overhead of the code stream.
  • the first-type patch, the second-type patch, and the third-type patch in each frame of point cloud can be arranged in the manner shown in FIG. 8, and then encoded in the order shown in FIG. 8 when encoding.
  • the auxiliary information of the patch to be encoded encoded into the code stream may also include identification information of the reference patch of the patch to be encoded.
  • the identification information of the reference patch is used to uniquely identify the reference patch of the patch to be encoded in the reference frame of the patch to be encoded.
  • the identification information of the reference patch may be the position number of the reference patch in the reference frame.
  • the patch to be coded is the first type 3 patch, then it can be coded as described above. If the patch to be coded is not the first type 3 patch, at this time, it can be based on the two-dimensional information of the first type 3 patch and For the three-dimensional information, the two-dimensional information and the three-dimensional information of the patch to be encoded are encoded into the code stream using the differential encoding method, and other information is also encoded in the manner described above.
  • using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream means: the two-dimensional information and the first
  • the difference between the two-dimensional information of the third-type patch and the difference between the three-dimensional information of the patch to be encoded and the first three-dimensional patch of the third type are encoded into the code stream.
  • the value of the first identifier can be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch, only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • S1101 Parse the first identifier of the patch to be decoded from the code stream.
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the first identifier of the patch[i-1] to be decoded and the auxiliary information of the patch[i-1] to be decoded are sequentially parsed, the first identifier of the patch[i] to be decoded and the auxiliary information of the patch[i] to be decoded , And the first identifier of the patch[i+1] to be decoded and the auxiliary information of the patch[i+1] to be decoded.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point clouds in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information
  • the patch to be decoded is the first type of patch
  • the auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.
  • a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream.
  • auxiliary information of the patch to be decoded for example, the two-dimensional information of the patch to be decoded, rotation transformation information and normal axis information
  • the position of the patch to be decoded can be obtained from the decoded point cloud and the position of the patch to be decoded in the current point cloud frame
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the same patch, and the obtained two-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and normal direction of the patch to be decoded Coordinate axis information.
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded can be obtained from the reference frame of the current point cloud frame, and the two-dimensional information, rotation transformation information and normal coordinate information of the reference patch can be obtained
  • the axis information is used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the reference patch of the patch to be decoded can be the patch whose position in the reference frame of the current point cloud frame is the same as the position of the patch to be decoded in the current point cloud frame .
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the manner shown in Figure 8.
  • the first identifier is the first value
  • the auxiliary information of the patch to be decoded encoded into the code stream may also include the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information parsed from the code stream may also include identification information of the reference patch of the patch to be decoded.
  • the reference patch of the patch to be decoded can be determined according to the identification information of the reference patch of the patch to be decoded.
  • the method of using differential decoding to parse the 3D information of the patch to be decoded from the code stream may specifically be: obtaining the 3D information of the patch whose position is the same as the position of the patch to be decoded in the current point cloud frame from the decoded point cloud. Or, obtain the three-dimensional information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame. Then, the obtained three-dimensional information is added with the difference of the three-dimensional information parsed from the code stream to obtain the three-dimensional information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and identification information of the reference patch of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the second type of patch that is encoded into the code stream during the encoding process includes two-dimensional information, three-dimensional information and identification information of the reference patch, but does not include the rotation transformation information and the normal coordinate axis information, so when the patch to be decoded When it is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner.
  • the reference patch of the patch to be decoded can be determined by the identification information of the reference patch of the patch to be decoded, and then the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded can be obtained. Then, the obtained two-dimensional information and three-dimensional information are respectively added with the two-dimensional information difference and the three-dimensional information difference parsed from the code stream, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the application embodiment does not limit this.
  • the first type patch, the second type patch, and the third type patch in each frame of point cloud in the current point cloud group can be arranged as shown in Figure 8.
  • the patch to be decoded is the current The first patch in the point cloud frame
  • the first type of patch, the second type of patch, and the third type of patch in each frame of point cloud in the current point cloud group are not arranged in the way shown in Figure 8, that is, the current point cloud group
  • the first identifier of the first patch in the point cloud of any frame in the point cloud group may be the first value, the second value, or the third value.
  • the first identifiers of multiple patches included in the current point cloud frame can be determined.
  • the multiple patches are all the patches included in the patch to be decoded. If the first identifiers of the multiple patches are all the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.
  • the first identifier is the third value
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are encoded by differential encoding.
  • Other auxiliary information is coded using fixed-length coding.
  • differential decoding can be used to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream
  • fixed-length decoding can be used to parse the to-be-decoded patch from the code stream. Decode the rotation transformation information and normal axis information of the patch.
  • the method of using differential decoding to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream is: parse the two-dimensional information and the first information of the patch to be decoded from the code stream.
  • the difference between the two-dimensional information of the third-type patch is added to the two-dimensional information of the first third-type patch to obtain the two-dimensional information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • the first identifier when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the patch to be decoded. Three-dimensional information.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information and information of the patch to be decoded. The identification information of the reference patch of the patch to be decoded.
  • the first identifier When the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, and Rotation transformation information and normal coordinate axis information. That is, for the first type of patch and the second type of patch, only part of the auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. In this way, the format of the parsed code stream can be simplified and the code can be reduced. Stream bit overhead, thereby improving decoding efficiency.
  • the embodiment of the present application can indicate different types of patches through the value of the first identifier, that is, it can distinguish between the first type of patch, the second type of patch, and the third type of patch, so there is no need to parse in the code stream.
  • the number of the first type of patch further reduces the bit overhead of the code stream and improves the decoding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100.
  • the method includes:
  • S1201 can refer to S1101 in the embodiment shown in FIG. 11, which will not be repeated in this embodiment of the present application.
  • S1202 Coding the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into the code stream.
  • the first-type patch has matching patches in all the point clouds in the current point cloud group, and the two-dimensional information, rotation transformation information, and information between the first-type patches in the current point cloud group
  • the normal coordinate axis information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch may also be the same.
  • the first identifier is the first value, it may indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream may also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identifier may be override_3d_shift_data_flag, that is, override_3d_shift_data_flag may be referred to as the third syntax element.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded.
  • the auxiliary information of the patch to be encoded may include three-dimensional information.
  • the specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used.
  • the syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier, see Table 1 and Table 3.
  • the information encoded into the code stream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the third identifier may be override_2d_shift_data_flag, that is, override_2d_shift_data_flag may be referred to as the second syntax element. Among them, the second identifier has already been described, so it will not be repeated here.
  • the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded can be Does not include two-dimensional information.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the two-dimensional information of the patch to be encoded is not the same as the two-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include Two-dimensional information.
  • the syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.
  • this article illustrates a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier and the third identifier, see Table 1 and Table 4 .
  • the two-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, not only the above-mentioned second and third marks can be used to indicate the coding Whether the information of the stream includes 2D information and 3D information of the patch to be decoded. It can also be indicated in other ways.
  • the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream, that is, when the first identifier is the first value , It can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include a fourth identifier.
  • the fourth identified syntax element may be override_patch_data_flag.
  • the fourth identifier is the eighth value, it may indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not included.
  • the fourth identifier is a ninth value
  • it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream.
  • the syntax element of the eighth value may be flase, and the syntax element of the ninth value may be true.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), and The offset of the encoded patch in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • each of the above identifiers can have two values, and the syntax elements of the two values can be true and false.
  • the meaning of each value can refer to the meaning of different values of the second identifier, the third identifier or the fourth identifier, which will not be repeated here.
  • this article exemplifies when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the fourth identifier, and the identifier used to indicate different two-dimensional information and three-dimensional information. Part of the grammatical structure, see Table 1 and Table 5.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to write the number of the first type of patches in the code stream, which reduces the bit overhead of the code stream.
  • the information encoded into the code stream may also include a second identifier to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information.
  • Information in this way, can further reduce the bit overhead of the code stream.
  • only part of the auxiliary information is encoded into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. In this case, the auxiliary information of the reference patch of the patch to be decoded can be used as the auxiliary information of the patch to be decoded.
  • the second identifier is the fifth value, it may indicate that the three-dimensional information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes the three-dimensional information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.
  • the information of the codestream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded can also be parsed from the code stream.
  • the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so the two-dimensional information of the reference patch of the patch to be encoded can be used as the two-dimensional information of the patch to be encoded.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • step 1202 when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are different, not only
  • the above-mentioned second identifier and third identifier can be used to indicate whether the information compiled into the code stream includes the two-dimensional information and three-dimensional information of the patch to be decoded. It can also be indicated in other ways, for example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream.
  • the fourth identifier of the patch to be decoded can be parsed from the code stream.
  • the fourth identifier is the eighth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not encoded.
  • the two-dimensional information and three-dimensional information of the reference patch of the patch are used as the two-dimensional information and three-dimensional information of the patch to be encoded.
  • the fourth identifier is the ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. Parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • the identifier used to indicate whether 2d_shift_u has been encoded into the code stream (override_2d_shift_u)
  • the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v)
  • the auxiliary information includes the two-dimensional information, the three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, so that there is no need to parse the number of the first type of patches in the code stream, and the bit overhead of the code stream is reduced. Moreover, when the first identifier is the first value, the second identifier of the patch to be decoded is parsed from the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream.
  • the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded.
  • the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream. Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.
  • the encoding device 1400 may include:
  • An auxiliary information encoding module 1402 configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information;
  • the first identifier is a second value to indicate that the patch to be encoded is a second type of patch
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded
  • the first identifier is a third value to indicate that the patch to be encoded is a third-type patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively, or the first value, the second value and the third value are binary 11, 10 and 00 respectively.
  • the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • the patch information acquisition module 1401 is used to execute the related content of step S901 of the embodiment shown in FIG. 9, and the auxiliary information encoding module 1402 is used to execute the related content of step S902 of the embodiment shown in FIG. 9.
  • the patch information acquisition module 1401 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1402 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words,
  • the function of the patch information acquisition module 1401 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG.
  • the function of the auxiliary information generating module 1402 may be implemented by the auxiliary information encoding module 109 in FIG. 2.
  • the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded
  • the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • the decoding apparatus 1500 may include:
  • the auxiliary information decoding module 1501 is used to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module 1501 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value, The auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, from the code stream Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module 1502 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the first value, the second value, and the third value are binary 10, 11, and 00 respectively, or the first value, the second value, and the third value are binary 11, 10, and 00, respectively.
  • auxiliary information decoding module 1501 is also used for:
  • the patch to be decoded is the first patch and the first identifier is the third value, it is determined that the current point cloud frame to which the patch to be decoded belongs is the first point cloud frame.
  • auxiliary information decoding module 1501 is also used for:
  • the first identifier of the patch to be decoded is parsed from the code stream; or, when the patch to be decoded is the first patch in the current point cloud frame, from the code stream Analyze multiple first identifiers of multiple patches included in the current point cloud frame, and the multiple first identifiers parsed include the first identifier of the patch to be decoded.
  • the auxiliary information decoding module 1501 is further configured to:
  • the reconstruction module 1502 is also used to:
  • the patch to be decoded is reconstructed.
  • the auxiliary information decoding module 1501 is further configured to:
  • the reconstruction module 1502 is also used to:
  • the patch to be decoded is reconstructed.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the auxiliary information decoding module 1501 is used to execute the related content of steps S1101-S1104 in the embodiment shown in FIG. 11, and the reconstruction module 1502 is used to execute the related content of step S1105 in the embodiment shown in FIG.
  • the auxiliary information decoding module 1501 in FIG. 15 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1502 in FIG. 15 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1501 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1502 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6.
  • this application is not limited to this.
  • the encoding apparatus 1600 may include:
  • the point cloud block patch information acquisition module 1601 is used to acquire auxiliary information of the patch to be encoded
  • An auxiliary information encoding module 1602 configured to encode auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded into a code stream;
  • the first identifier is a first value to indicate that the patch to be encoded is a first-type patch
  • the information encoded into the code stream also includes a second identifier, where the second identifier is a fourth value to indicate the three-dimensional information of the patch to be encoded
  • the code stream is not coded; or, the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be coded has been coded into the code stream, and the auxiliary information of the patch to be coded includes three-dimensional information;
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the patch information acquisition module 1601 is used to execute the related content of step S1201 in the embodiment shown in FIG. 12, and the auxiliary information encoding module 1602 is used to execute the related content of step S1202 in the embodiment shown in FIG.
  • the patch information acquisition module 1601 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in FIG. 2
  • the function of the patch information acquisition module 1601 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2
  • the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in FIG.
  • the point cloud block information generation module 101 is used to obtain three-dimensional information, normal coordinate axis information, first identification and second identification of the patch to be encoded, and the packaging module 102 is used to obtain rotation transformation information of the patch to be encoded , Refer to patch identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • FIG. 17 is a schematic structural diagram of a second point cloud decoding apparatus provided by an embodiment of this application.
  • the decoding apparatus 1700 may include:
  • the auxiliary information decoding module 1701 is used to parse the first identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module 1701 is also used to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value; when the second identifier is the fourth value, to add the auxiliary information of the reference patch of the patch to be decoded
  • the information is used as auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the first identifier is the second value Analyze the auxiliary information of the patch to be decoded from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, the auxiliary information Parse the auxiliary information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded;
  • the reconstruction module 1702 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information decoding module 1701 is used to execute related content of steps S1301 to S1305 of the embodiment shown in FIG. 13, and the reconstruction module 1702 is used to execute related content of step S1306 of the embodiment shown in FIG. 13.
  • the auxiliary information decoding module 1701 in FIG. 17 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1702 in FIG. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in FIG. 6.
  • this application is not limited to this.
  • FIG. 18 is a schematic block diagram of an implementation manner of a decoding device 1800 used in an embodiment of the present application.
  • the obtaining apparatus 1800 may include a processor 1801, a memory 1802, and a bus system 1803.
  • the processor 1801 and the memory 1802 are connected through a bus system 1803, the memory 1802 is used to store instructions, and the processor 1801 is used to execute instructions stored in the memory 1802 to execute various point cloud encoding or decoding methods described in this application , Especially the method of filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.
  • the processor 1801 may be a central processing unit (CPU), and the processor 1801 may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device can also be used as the memory 1802.
  • the memory 1802 may include code and data 18021 accessed by the processor 1801 using the bus 1803.
  • the memory 1802 may further include an operating system 18023 and an application program 18022.
  • the application program 18022 includes at least one of the point cloud encoding or decoding methods described in this application (especially the point cloud encoding and decoding methods described in this application) that allows the processor 1801 to execute program.
  • the application 18022 may include applications 1 to N, which further include a point cloud encoding or decoding application (referred to as a point cloud decoding application) that executes the point cloud encoding or decoding method described in this application.
  • the bus system 1803 may also include a power bus, a control bus, and a status signal bus. However, for clear description, various buses are marked as the bus system 1803 in the figure.
  • the encoding apparatus 1800 may further include one or more output devices, such as a display 1804.
  • the display 1804 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input.
  • the display 1804 may be connected to the processor 1801 via a bus 1803.
  • the decoding device 1800 can execute the point cloud encoding method in this application, and can also execute the point cloud decoding method in this application.
  • the computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (for example, according to a communication protocol) .
  • computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves.
  • Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is appropriately called a computer-readable medium.
  • any connection is appropriately called a computer-readable medium.
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media.
  • magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
  • DSP digital signal processors
  • ASIC application-specific integrated circuits
  • FPGA field programmable logic arrays
  • processor may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein.
  • the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec.
  • the technology may be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.
  • the technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets).
  • ICs integrated circuits
  • a set of ICs for example, chipsets.
  • Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units.
  • various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention concerne un procédé de codage de nuage de points, un procédé de décodage de nuage de points, un appareil et un support de stockage relevant du domaine du traitement de données. Le procédé comprend les étapes consistant à : obtenir des informations auxiliaires d'un correctif devant être codé ; et coder dans un flux de code les informations auxiliaires et un premier identifiant du correctif devant être codé. La valeur du premier identifiant peut être une première valeur, une deuxième valeur ou une troisième valeur. Des valeurs différentes indiquent différents types de correctifs. Par conséquent, le premier identifiant permet de distinguer les différents types de correctifs. Pour différents types de correctifs, le contenu intégré dans les informations auxiliaires codées dans le flux de code peut être différent. Ainsi le format du codage dans le flux de code peut-il être simplifié, ce qui réduit le surdébit binaire du flux de code et, par conséquent, accroît l'efficacité de codage.
PCT/CN2020/080140 2019-03-19 2020-03-19 Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage WO2020187283A1 (fr)

Priority Applications (6)

Application Number Priority Date Filing Date Title
EP20774250.3A EP3934250A4 (fr) 2019-03-19 2020-03-19 Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage
SG11202110220RA SG11202110220RA (en) 2019-03-19 2020-03-19 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
BR112021018435A BR112021018435A2 (pt) 2019-03-19 2020-03-19 Método e aparelho de codificação de nuvem de pontos, método e aparelho de decodificação de nuvem de pontos, e meio de armazenamento
JP2021556407A JP7453992B2 (ja) 2019-03-19 2020-03-19 点群符号化方法及び装置、点群復号方法及び装置、並びに記憶媒体
KR1020217033214A KR20210134401A (ko) 2019-03-19 2020-03-19 포인트 클라우드 인코딩 방법 및 장치, 포인트 클라우드 디코딩 방법 및 장치, 및 저장 매체
US17/477,812 US20220007037A1 (en) 2019-03-19 2021-09-17 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910210181 2019-03-19
CN201910210181.3 2019-03-19
CN201910214557.8 2019-03-20
CN201910214557.8A CN111726616B (zh) 2019-03-19 2019-03-20 点云编码方法、点云解码方法、装置及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/477,812 Continuation US20220007037A1 (en) 2019-03-19 2021-09-17 Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2020187283A1 true WO2020187283A1 (fr) 2020-09-24

Family

ID=72519591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/080140 WO2020187283A1 (fr) 2019-03-19 2020-03-19 Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage

Country Status (1)

Country Link
WO (1) WO2020187283A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN108322742A (zh) * 2018-02-11 2018-07-24 北京大学深圳研究生院 一种基于帧内预测的点云属性压缩方法
CN108833927A (zh) * 2018-05-03 2018-11-16 北京大学深圳研究生院 一种基于删除量化矩阵中0元素的点云属性压缩方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN108322742A (zh) * 2018-02-11 2018-07-24 北京大学深圳研究生院 一种基于帧内预测的点云属性压缩方法
CN108833927A (zh) * 2018-05-03 2018-11-16 北京大学深圳研究生院 一种基于删除量化矩阵中0元素的点云属性压缩方法

Similar Documents

Publication Publication Date Title
JP7057453B2 (ja) 点群符号化方法、点群復号化方法、符号器、及び復号器
WO2020151496A1 (fr) Procédé et appareil de codage/décodage de nuage de points
JP2022517060A (ja) 点群符号化構造
US11388442B2 (en) Point cloud encoding method, point cloud decoding method, encoder, and decoder
US11961265B2 (en) Point cloud encoding and decoding method and apparatus
WO2020011265A1 (fr) Procédé de codage et de décodage de nuage de points et codeur-décodeur
CN110944187B (zh) 点云编码方法和编码器
WO2020147379A1 (fr) Procédé et dispositif de filtrage de nuage de points et support de stockage
JP2021518694A (ja) 三次元物体を表すポイントクラウドを符号化／復号する方法及び装置
US20220007037A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
JP2021518681A (ja) 三次元物体を表すポイントクラウドを符号化／復号する方法及び装置
CN111726615A (zh) 点云编解码方法及编解码器
WO2020187283A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage
WO2020220941A1 (fr) Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage
US20230156229A1 (en) Methods and apparatus to signal enabled views per atlas in immersive video
WO2020015517A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points et décodeur
US20240129537A1 (en) Method and apparatus for signaling cmaf switching sets in isobmff
WO2020187191A1 (fr) Procédé de codage et de décodage de nuage de points et codec
WO2023071462A1 (fr) Procédé et appareil d'encodage de nuage de points, procédé et appareil de décodage de nuage de points, dispositif, support de stockage et produit de programme
CN114760525A (zh) 视频生成及播放方法、装置、设备、介质
CN117751387A (zh) 面元网格连通性编码
US20120263390A1 (en) Decoding a quilted graphic 2d image representation into a digital asset

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20774250

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021556407

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112021018435

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 2020774250

Country of ref document: EP

Effective date: 20210930

ENP Entry into the national phase

Ref document number: 20217033214

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 112021018435

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20210916