WO2020220941A1 - Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage - Google Patents

Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage Download PDF

Info

Publication number
WO2020220941A1
WO2020220941A1 PCT/CN2020/083492 CN2020083492W WO2020220941A1 WO 2020220941 A1 WO2020220941 A1 WO 2020220941A1 CN 2020083492 W CN2020083492 W CN 2020083492W WO 2020220941 A1 WO2020220941 A1 WO 2020220941A1
Authority
WO
WIPO (PCT)
Prior art keywords
patch
information
point cloud
decoded
identifier
Prior art date
Application number
PCT/CN2020/083492
Other languages
English (en)
Chinese (zh)
Inventor
蔡康颖
张德军
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020220941A1 publication Critical patent/WO2020220941A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding

Definitions

  • This application relates to the field of data processing technology, and in particular to a point cloud encoding method, point cloud decoding method, device and storage medium.
  • This application provides a point cloud encoding method, point cloud decoding method, device and storage medium, which can solve the problem of how to effectively encode and decode point clouds in related technologies.
  • the technical solution is as follows:
  • a point cloud encoding method includes: obtaining auxiliary information of a patch to be encoded (point cloud block) and a first identifier of the patch to be encoded; when the first identifier is a first value, Encode the auxiliary information of the patch and the first identifier into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; or, when the first identifier is the first value, encode the auxiliary information of the patch to be encoded into the code Stream, the patch to be coded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate that the patch to be coded is the first type of patch, and the auxiliary information of the patch to be coded includes three-dimensional information.
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier is the first value
  • the first type of patch can be a global matching patch in the current point cloud group.
  • the auxiliary information of the patch to be encoded includes three-dimensional information. Including two-dimensional information, rotation transformation information and normal coordinate axis information.
  • the first type of patch exists in the current point cloud group
  • the number of the first type of patch in the current point cloud group can be the same in each point cloud frame, and when the number of the first type of patch is greater than 1
  • the first-type patches in each point cloud frame in the current point cloud group can be arranged in the same order, so when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame.
  • the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame.
  • the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting
  • the first identifier of the encoded patch is encoded into the code stream, so that the format of the encoded code stream can be simplified.
  • the auxiliary information of the patch to be coded may include three-dimensional information, but not two-dimensional information, rotation transformation information and normal direction. Coordinate axis information.
  • the patches in each point cloud frame can be arranged in the order of the first type of patch, the second type of patch and the third type of patch, and then the first type of patch and the second type of patch are also used when encoding.
  • the type patch and the third type patch are coded sequentially. So if there is a first-type patch in the current point cloud group, the first patch in each point cloud frame is the first-type patch.
  • the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first-type patches is greater than 1, the number of the first type of patches in each point cloud frame in the current point cloud group
  • One type of patches can be arranged in the same order, that is, there is a matching relationship between the first type of patches at the same position in each point cloud frame.
  • the first identification of all patches included in the first point cloud frame can be Set to the third value, the patches in other point cloud frames are still arranged in the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, when the first identifier of the patch to be encoded is the first value, it can be determined that the current point cloud frame is the non-first point cloud frame in the current point cloud group.
  • the current point cloud frame may be the Kth point cloud frame in the current point cloud group, and K is an integer greater than or equal to 3. That is, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the manner of S902.
  • the point cloud frames in the current point cloud group can be numbered according to two numbering methods.
  • the first numbering method point cloud frame 0, point cloud frame 1, point cloud frame 2...;
  • the second numbering method point cloud frame 1, point cloud frame 2, point cloud frame 3... Therefore, the above Kth point cloud frame can refer to the point cloud frame 2 in the first numbering method and any point cloud frame after the point cloud frame 2, or it can refer to the point cloud frame 3 and the point cloud frame in the second numbering method.
  • the Kth point cloud frame can refer to the third point cloud frame in the actual current point cloud group and the point cloud frames after the third.
  • the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the above manner, and for the second point cloud frame in the current point cloud group
  • Point cloud frames can be encoded in the following two ways:
  • the first possible implementation manner if the current point cloud frame is the second point cloud frame in the current point cloud group, then, when the first identifier of the patch to be encoded is the first value, and the patch to be encoded is the current point cloud
  • the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded can be incorporated into the code stream.
  • the auxiliary information of the patch to be coded can be encoded into the code stream instead of the patch to be coded
  • the first identifier is incorporated into the code stream.
  • the number of first-type patches in the current point cloud group can be obtained.
  • the operation of encoding the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: editing the first identifier of the patch to be encoded Enter the code stream, and then encode the number of first-type patches in the current point cloud group into the code stream, and finally, encode the auxiliary information of the patch to be encoded into the code stream. That is, the number of the first-type patch in the current point cloud group is compiled into the code stream after the first identifier of the patch to be encoded.
  • the fixed-length encoding method can be used to encode the number of patches of the first type into the code stream, and the variable-length encoding method can also be used to encode the number of patches of the first type into the code stream. Not limited.
  • the current point cloud frame is the second point cloud frame in the current point cloud group
  • the first value of the patch to be coded can be set
  • An identification and auxiliary information are compiled into the code stream. That is, the first identification and auxiliary information of all the first-type patches in the second point cloud frame need to be encoded into the code stream, so that the number of the first-type patches in the current point cloud group can be indicated without The number of the first type patch in the current point cloud group is compiled into the code stream.
  • the number of the first-type patch in the current point cloud group can be determined through the second point cloud frame.
  • the first patch that is, the first identifier of the first first type patch can be coded It is not necessary to encode the first identifier of the non-first first-type patch into the code stream.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame
  • the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame only the first identifier of the first type patch can be compiled into the code stream, instead of the non-first point cloud frame.
  • the first identifier of the first type patch is encoded into the code stream, which improves the coding efficiency and reduces the bit overhead of the code stream.
  • the first patch in the current point cloud frame can correspond to an identification information, which can also be called an index, and this identification information can uniquely indicate the first patch in the current point cloud frame patch, and the identification information of the first patch can usually be incorporated into the code stream.
  • an identification information which can also be called an index
  • this identification information can uniquely indicate the first patch in the current point cloud frame patch
  • the identification information of the first patch can usually be incorporated into the code stream.
  • a counter can be used to determine which patch is the current point cloud frame. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited.
  • the fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.
  • differential encoding to encode the three-dimensional information of the patch to be encoded into the code stream refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.
  • the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream.
  • the first identifier when the first identifier is the second value, it indicates that the patch to be encoded is a second type of patch.
  • the second type of patch can be a local matching patch in the current point cloud group.
  • the auxiliary information of the patch to be encoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be encoded.
  • the first identifier is the third value, it indicates that the patch to be coded is a third type of patch.
  • the third type of patch can be a non-matching patch in the current point cloud group.
  • the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, Rotation transformation information and normal coordinate axis information.
  • the second type of patch has a matching relationship in the reference frame of the current point cloud frame, and the rotation transformation information and normal coordinate axis information between the second type of patch and its reference patch are the same, and the two-dimensional information and the three-dimensional information It may be different.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value
  • the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the two-dimensional information of the patch to be encoded may include the offset (2d_shift_u) of the patch to be encoded in the current point cloud frame occupancy map along the u axis, and the patch to be encoded in the current point cloud frame occupancy map along the v axis
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be compiled into the code stream according to the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method encodes the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream, and a more accurate predictive differential method can also be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream.
  • the specific encoding method is not used. limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal axis information and the first identifier of the patch to be encoded into the code stream, or variable
  • the long encoding method encodes the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and first identifier of the patch to be encoded into the code stream, which is not limited in the embodiment of the present application.
  • using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be encoded, and The difference between the three-dimensional information of the coded patch and the three-dimensional information of the reference patch of the patch to be coded is encoded into the code stream.
  • the first identifier of the first first-type patch in the current point cloud frame the first identifier of each second-type patch, and the first identifier of each third-type patch in the current point cloud frame
  • the logo needs to be programmed into the code stream.
  • the auxiliary information of each patch included in the current point cloud frame also needs to be compiled into the code stream. There may be the following three possible implementation methods for the encoding method of the first identification and auxiliary information.
  • the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches.
  • the auxiliary information of each patch included in the current point cloud frame can be individually packaged and compiled into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Then, the auxiliary information of each patch included in the current point cloud frame is packaged one by one and compiled into the code stream.
  • the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately
  • the auxiliary information of each patch included in the current point cloud frame can also be packaged separately and compiled into the code stream. For example, you can first encode the first identifier of the first patch in the current point cloud frame into the code stream, and then encode the auxiliary information of the first patch into the code stream. Then, the auxiliary information of other first-type patches in the current point cloud frame is compiled into the code stream one by one. Finally, for the remaining patches, that is, the second type of patch and the third type of patch, the coding can be performed in a way that the first identifier is first compiled, and then the auxiliary information is compiled.
  • the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches.
  • the same type of information in the auxiliary information of each patch can be packaged together and compiled into the code stream. Different types of information in the auxiliary information of each patch can be individually packaged and compiled into the code stream.
  • the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream.
  • the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information
  • some auxiliary information of patches includes two-dimensional information or three-dimensional information
  • some auxiliary information of patches does not include two-dimensional information or
  • the rotation transformation information and normal coordinate axis information in the auxiliary information of each patch are packaged together and compiled into the code stream.
  • the two-dimensional information and three-dimensional information in the auxiliary information of each patch are individually packaged and compiled into the code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.
  • the patches in each point cloud frame in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch.
  • the patch to be coded is the first third type of patch
  • the fixed-length encoding method may be used for encoding, or the variable-length encoding method may be used, which is not limited in the embodiment of the present application.
  • the 2D information and 3D information of the patch to be coded can be encoded into the code stream using the differential encoding method, and other The information is also encoded using the fixed-length encoding method described above.
  • the different values of the first identifier indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, and for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • a point cloud decoding method includes: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, parsing the first identifier of the patch to be decoded from the code stream; When the first identifier is the first value, the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of patches of the first type in the reference point cloud frame of the current point cloud frame, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; Decode the auxiliary information of the patch and reconstruct the patch to be decoded.
  • the to be decoded can be parsed from the code stream The first ID of the patch.
  • the code stream includes the identification information of the patch to be decoded, and if the identification information of the patch to be decoded is included, it can be determined that the patch to be decoded is the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, it can be determined that the patch to be decoded is the non-first patch in the current point cloud frame. At this time, the number of the patch to be decoded in the current point cloud frame can be determined according to the counter. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point cloud frames in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the number of patches of the first type can be parsed from the code stream by a method that matches the encoding end.
  • fixed-length decoding or variable-length decoding can be used to parse the number of patches of the first type from the code stream.
  • the first identifier when the first identifier is the first value, it can indicate that the current point cloud frame is the non-first point cloud frame in the current point cloud group.
  • the patch to be decoded can be decoded through the following two possible implementation methods.
  • the first possible implementation is the first identification of the first patch in the second point cloud frame in the current point cloud group, and the number of the first type of patch included in each point cloud frame in the current point cloud group Code stream together.
  • the first patch to be decoded can be parsed from the code stream. Identification, when the first identification is the first value, the number of the first-type patches included in each point cloud frame in the current point cloud group is analyzed from the code stream. Then parse the auxiliary information of the patch to be decoded from the code stream.
  • the first identifiers of the first type patch included in the second point cloud frame in the current point cloud group are all encoded into the code stream.
  • the first identifier of the patch to be decoded can be parsed from the code stream.
  • the first identifier is the first value
  • the patch to be decoded is parsed from the code stream. Supplementary information.
  • the first identifier of the non-first first-type patch is not encoded into the code stream, so when the patch to be decoded is the nth in the current point cloud frame
  • n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first first-type patch in the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information
  • the patch to be decoded is the first type of patch
  • the auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.
  • a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream.
  • the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, and then according to the The identification information of the reference frame determines the reference frame of the current point cloud frame, and then obtains the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame, and refers to the patch
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch.
  • the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame.
  • the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame
  • the auxiliary information of the patch is determined.
  • the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can be obtained from the decoded point cloud, and the obtained two-dimensional information,
  • the rotation transformation information and the normal coordinate axis information are used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the to-be-decoded patch to assist
  • the information includes two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the decoded patch.
  • the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded A patch with the same position in the current point cloud frame.
  • the counter can be used to determine the number of patches in the current point cloud frame to be decoded.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame
  • the patch to be decoded can be determined It is the non-first first-type patch in the current point cloud frame, so that the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the number of first-type patches included in each point cloud frame in the current point cloud group can be determined in two ways.
  • the first possible implementation is that after decoding the first identifier of the first patch in the second point cloud frame in the current point cloud group, each point in the current point cloud group can be directly parsed from the code stream The number of first-type patches included in the cloud frame.
  • the second possible implementation manner after decoding the first identifiers of all patches of the second point cloud frame in the current point cloud group, the number of patches with the first identifier as the first value can be determined, so as to determine The number of first-type patches included in the second point cloud frame, that is, the number of first-type patches included in each point cloud frame in the current point cloud group. That is, the number of first-type patches included in each point cloud frame in the current point cloud group can be determined through the second point cloud frame.
  • the second point cloud frame in the current point cloud group can be used as the reference point cloud frame of the third point cloud frame
  • the third point cloud frame can be used as the reference point cloud frame of the fourth point cloud frame Point cloud frame, and so on.
  • the third point cloud frame and the point cloud frames after the third can determine whether n in the nth patch in the current point cloud frame is greater than 1 and based on the number of the first type of patch in the reference point cloud frame
  • the number of the first-type patch in the reference point cloud frame of the current point cloud frame is less than or equal to, so as to determine whether the patch to be decoded is the first-type patch.
  • the first identifier of the first patch in the current point cloud frame, and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately and compiled into the code stream, Can be packaged together and compiled into code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for patches that take the second value or the third value other than the first patch, the first identifier of the patch to be decoded can also be parsed from the code stream.
  • the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be parsed in sequence.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream
  • the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packaged and compiled into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, you can The code stream parses the same type of information in the auxiliary information of each patch included in the current point cloud frame. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of first-type patches in the reference point cloud frame of the current point cloud frame, the to-be-decoded patch is parsed from the code stream The first ID of the patch.
  • the first identifier is encoded into the code stream, so when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current
  • the point cloud frame refers to the number of the first-type patch in the point cloud frame, it indicates that the patch to be decoded is a second-type patch or a third-type patch.
  • the first identifier of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, the three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. That is, for the second type of patch, two-dimensional information and three-dimensional information can be parsed from the code stream, while the rotation transformation information and the normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream
  • the rotation transformation information and the normal coordinate axis information also reduce the bit overhead of the code stream and improve the decoding efficiency at the same time.
  • reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • a method that matches the encoding end can be used for decoding.
  • fixed-length decoding can be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded from the code stream.
  • the variable-length decoding method can also be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded from the code stream, which is not limited in the embodiment of the present application.
  • the auxiliary information of the first third-type patch is coded according to fixed-length coding or variable-length coding
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by differential coding
  • other auxiliary information is coded Encode with fixed-length encoding.
  • the two-dimensional information and three-dimensional information of the patch to be decoded are parsed from the code stream by differential decoding, and fixed
  • the long decoding method parses the rotation transformation information and normal coordinate axis information of the patch to be decoded from the code stream.
  • the non-first third type of patch two-dimensional information and three-dimensional information can be parsed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so as to avoid
  • the bit overhead of the code stream is also reduced, and the decoding efficiency is improved.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the application embodiment does not limit this.
  • the first type patch, the second type patch, and the third type patch in each point cloud frame in the current point cloud group can be arranged in sequence.
  • the patch to be decoded is the first in the current point cloud frame A patch
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained, which can simplify the format of the code stream. , Reduce the bit overhead of the code stream, thereby improving the efficiency of decoding.
  • a point cloud coding method includes: obtaining auxiliary information of a point cloud block patch to be coded and a first identifier of the patch to be coded; when the first identifier is a first value, the patch to be coded.
  • the auxiliary information, the first identifier, and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; or, when the first identifier is the first value, the patch to be encoded
  • the auxiliary information and the second identifier of the patch to be encoded are encoded into the code stream, the patch to be encoded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate that the patch to be encoded is the first type of patch ;
  • the second identifier is the fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; or the second identifier is the fifth value to indicate
  • the first type of patch has matching patches in all point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis between the first type of patches in the current point cloud group
  • the information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch can also be the same.
  • the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identification may be override_3d_shift_data_flag.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value
  • the specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used.
  • the syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the patch to be encoded is the first patch in the current point cloud frame.
  • the auxiliary information, the first identifier and the second identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, When the patch to be coded is the non-first patch in the current point cloud frame, it indicates that the patch to be coded is the non-first type 1 patch in the current point cloud frame, and the auxiliary information and the second identifier of the patch to be coded can be coded Instead of coding the first identifier of the patch to be coded into the code stream, the format of the code stream can be simplified.
  • the second identifier is an identifier used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. Information, in this way, can further reduce the bit overhead of the code stream, thereby improving the coding efficiency.
  • a point cloud decoding method includes: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, parsing the first identifier of the patch to be decoded from the code stream; When an identifier is the first value, analyze the second identifier of the patch to be decoded from the code stream; when the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; When the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, the second identifier of the patch to be decoded is parsed
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the 3D information of the patch to be decoded is the same as the 3D information of the reference patch of the patch to be decoded.
  • the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded. Specifically, the auxiliary information of the reference patch of the patch to be decoded may be used as the auxiliary information of the patch to be decoded.
  • the second identifier When the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.
  • the second identifiers of the non-first first-type patch in the current point cloud frame are all encoded into the bitstream, when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current
  • the point cloud frame refers to the number of the first-type patch in the point cloud frame, that is, when the patch to be decoded is not the first first-type patch
  • the second identifier of the patch to be decoded can be parsed from the code stream.
  • the patch to be decoded is parsed from the code stream
  • the first identifier is the first value, it indicates that the patch to be decoded is the first patch of the first type.
  • the second identifier of the patch to be decoded can be parsed from the code stream.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the second identifier of the patch to be decoded can be parsed from the code stream.
  • the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded.
  • the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.
  • a point cloud coding method comprising: obtaining auxiliary information of a point cloud block to be coded and a first identifier of the patch to be coded; when the first identifier is a first value, the patch to be coded When the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream; where the first identifier is the first value to indicate The patch to be coded is the first type of patch; the first identifier is the second value to indicate that the patch to be coded is the second type of patch, and the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information and the reference patch of the patch to be coded Identification information; where the first identification is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first identifier When the first identifier is the first value, it indicates that the patch to be encoded is the first type of patch.
  • the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and two-dimensional information between the first-type patches in the current point cloud group , 3D information, rotation transformation information and normal coordinate axis information are all the same. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is encoded into the code stream instead of the auxiliary information of the patch to be coded. Code stream.
  • the first identifier of the patch to be encoded is the first value
  • the first identifier of the patch to be coded can be encoded into the code stream, instead of coding the auxiliary information of the patch to be coded into the code stream, so that the format of the code stream can be simplified.
  • the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches.
  • auxiliary information encoded in the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • a point cloud decoding method includes: parsing a first identifier of a point cloud block patch to be decoded from a code stream; when the first identifier is a first value, according to the reference patch of the patch to be decoded
  • the auxiliary information determines the auxiliary information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; when the first identifier is the second value, it is parsed from the code stream The auxiliary information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, parse the patch to be decoded from the code stream
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information; according to the auxiliary information of the patch to be decoded, the patch to be decoded is reconstructed.
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • the reference patch of the patch to be decoded can be Auxiliary information to obtain 2D information, 3D information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, and then from the reference frame of the current point cloud frame Obtain the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded, and use the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch as the information to be decoded Patch's two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch.
  • the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame.
  • the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame
  • the auxiliary information of the patch is determined.
  • the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc.
  • the position of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can also be obtained from the decoded point cloud.
  • Two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information, and the obtained two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information.
  • the reference patch of the patch to be decoded can be the reference of the current point cloud frame
  • the patch whose position in the frame is the same as the position of the patch to be decoded in the current point cloud frame.
  • the auxiliary information of the second type of patch that is incorporated into the code stream during the encoding process includes two-dimensional information, three-dimensional information, and identification information of the reference patch, but does not include rotation transformation information and normal coordinate axis information
  • the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and information of the patch to be decoded.
  • Normal axis information At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
  • auxiliary information can be derived or obtained.
  • the auxiliary information can also be derived or obtained. In this way, the format of the parsing code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.
  • a point cloud encoding device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding module , Used to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the first value, the patch to be encoded is the first patch in the current point cloud frame; or the auxiliary information encoding module, using When the first identifier is the first value, the auxiliary information of the patch to be encoded is encoded into the code stream, and the patch to be encoded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate the The coded patch is the first type of patch, and the auxiliary information of the patch to be coded includes three-dimensional information.
  • the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.
  • the auxiliary information encoding module is also used for:
  • the first identifier is the second value or the third value
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information encoding module is also used for:
  • the first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches are excluding the patch to be encoded
  • the remaining first identifier besides the first identifier is the second value or the third value.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively,
  • the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  • the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • a point cloud decoding device includes: an auxiliary information decoding module, configured to parse the to-be-decoded point cloud block patch from the current point cloud frame when the patch to be decoded is the first patch in the current point cloud frame The first identifier of the patch; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded; the auxiliary information
  • the decoding module is also used for when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, from
  • the auxiliary information of the patch to be decoded is parsed in the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.
  • the auxiliary information decoding module is also used for:
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patch in the reference point cloud frame of the current point cloud frame, parse the patch to be decoded from the code stream First logo
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded;
  • the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the auxiliary information decoding module is also used for:
  • the first identifiers of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed include the patch to be decoded A first identifier, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively,
  • the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  • the current point cloud frame is the first point cloud frame.
  • the auxiliary information decoding module is further configured to:
  • the reconstruction module is also used to:
  • the patch to be decoded is reconstructed.
  • the auxiliary information decoding module is also used to:
  • the reconstruction module is also used to:
  • the patch to be decoded is reconstructed.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • a point cloud encoding device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding module , Used to encode the auxiliary information of the patch to be encoded, the first identifier and the second identifier of the patch to be encoded into the code stream when the first identifier is the first value, the patch to be encoded is the first patch in the current point cloud frame Or auxiliary information encoding module, when the first identifier is the first value, the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded into the code stream, and the patch to be encoded is the non-information in the current point cloud frame
  • the first patch; where the first identifier is the first value to indicate that the patch to be encoded is the first type of patch; where the second identifier is the fourth value to indicate that the three-dimensional information of the patch to be
  • a point cloud decoding device includes: an auxiliary information decoding module for analyzing the to-be-decoded point cloud block patch from the current point cloud frame when the patch to be decoded is the first patch in the current point cloud frame The first identifier of the patch; the auxiliary information decoding module is also used to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value; the auxiliary information decoding module is also used to When the identifier is the fourth value, the auxiliary information of the patch to be decoded is determined according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information Including the three-dimensional information of the patch to be decoded; the auxiliary information decoding module is also used when the patch to be decoded is the nth patch in the current point cloud frame
  • a point cloud encoding device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding The module is used to encode the first identifier of the patch to be encoded into the code stream when the first identifier is the first value; the auxiliary information encoding module is also used to when the first identifier is the second value or the third value, The auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream; the first identifier is the first value to indicate that the patch to be encoded is a first type of patch; the first identifier is the second value to indicate the patch to be encoded It is the second type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded; the first identifier is a third value to indicate that the patch to be encoded is a third-type patch,
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • a point cloud decoding device includes: an auxiliary information decoding module for parsing the first identifier of the point cloud block patch to be decoded from the code stream; the auxiliary information decoding module is also used for When the first identifier is the first value, the auxiliary information of the patch to be decoded is determined according to the auxiliary information of the reference patch of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis. Information; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the second value.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information and the patch to be decoded Reference patch identification information; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the third value.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information Information, rotation transformation information, and normal coordinate axis information; the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information decoding module is also used to:
  • the reconstruction module is also used to:
  • the patch to be decoded is reconstructed.
  • a device for encoding point cloud data may include a memory and an encoder.
  • the memory is used to store point cloud data.
  • the encoder is used to implement the point cloud coding method provided by the first aspect or any possible design of the first aspect, or the encoder is used to implement the points provided by the third aspect or any possible design of the third aspect.
  • the cloud coding method, or the encoder is used to execute the point cloud coding method provided by any possible design of the fifth aspect or the fifth aspect.
  • a device for decoding point cloud data may include a memory and a decoder.
  • the memory is used to store point cloud data in the form of a code stream.
  • the decoder is used to implement the point cloud decoding method provided by the foregoing second aspect or any possible design of the second aspect, or the decoder is used to implement the points provided by any possible design of the foregoing fourth aspect or the fourth aspect
  • the cloud decoding method, or the decoder is used to execute the point cloud decoding method provided by the sixth aspect or any one of the possible designs of the sixth aspect.
  • an encoding device including: a memory and a processor; wherein the memory is used for storing program code; the processor is used for calling the program code to execute the first aspect or the third aspect or the fifth aspect.
  • Point cloud coding method provided by the aspect.
  • a decoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the second aspect or the fourth aspect or the sixth aspect.
  • the point cloud decoding method provided by the aspect.
  • the present application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to execute the first aspect and its possible designs, or the third aspect and its possible designs, Or any point cloud coding method provided by the fifth aspect and its possible designs.
  • This application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to execute the second aspect and its possible designs, or the fourth aspect and its possible designs, or Any point cloud decoding method provided by the sixth aspect and its possible designs.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the first aspect or the first aspect, or execute the third aspect or the third aspect Any possible design, or execute part or all of the steps of the point cloud coding method provided by the fifth aspect or any possible design of the fifth aspect.
  • This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the second aspect or the second aspect, or execute the fourth aspect or the fourth aspect Any possible design, or execute part or all of the steps of the point cloud decoding method provided by the sixth aspect or any possible design of the sixth aspect.
  • the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame.
  • the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting
  • the first identifier of the encoded patch is encoded into the code stream, so that the format of the encoded code stream can be simplified. Further reduce the bit overhead of the code stream, thereby improving the coding efficiency.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application;
  • Fig. 2 is a schematic block diagram of an encoder that can be used in an embodiment of the present application
  • Fig. 3 is a schematic diagram of a point cloud applicable to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a point cloud patch applicable to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application.
  • Fig. 6 is a schematic block diagram of a decoder that can be used in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a frame group that can be used in an embodiment of the present application.
  • FIG. 8 is a schematic diagram of the first current point cloud group provided by an embodiment of this application.
  • FIG. 9 is a flowchart of the first point cloud encoding method provided by an embodiment of the application.
  • FIG. 10 is a schematic diagram of a second current point cloud group provided by an embodiment of this application.
  • FIG. 11 is a flowchart of the first point cloud decoding method provided by an embodiment of this application.
  • FIG. 12 is a flowchart of a second point cloud encoding method provided by an embodiment of this application.
  • FIG. 13 is a flowchart of a second point cloud decoding method provided by an embodiment of this application.
  • FIG. 14 is a flowchart of a third point cloud encoding method provided by an embodiment of this application.
  • FIG. 16 is a schematic block diagram of a first encoding device provided by an embodiment of this application.
  • FIG. 17 is a schematic block diagram of a first decoding device provided by an embodiment of this application.
  • FIG. 18 is a schematic block diagram of a second encoding device provided by an embodiment of this application.
  • FIG. 19 is a schematic block diagram of a second decoding apparatus provided by an embodiment of this application.
  • FIG. 20 is a schematic block diagram of a third encoding device provided by an embodiment of this application.
  • FIG. 21 is a schematic block diagram of a third decoding apparatus provided by an embodiment of this application.
  • FIG. 22 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application.
  • the term "point cloud decoding” or “decoding” may generally refer to point cloud encoding or point cloud decoding.
  • the point cloud decoding system includes a source device 10, a destination device 20, a link 30 and a storage device 40.
  • the source device 10 can generate coded point cloud data. Therefore, the source device 10 may also be referred to as a point cloud encoding device.
  • the destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device.
  • the link 30 can receive the coded point cloud data generated by the source device 10 and can transmit the coded point cloud data to the destination device 20.
  • the storage device 40 can receive the coded point cloud data generated by the source device 10, and can store the coded point cloud data, so that the destination device 20 can directly obtain the coded point cloud from the storage device 40 data.
  • the storage device 40 may correspond to a file server or another intermediate storage device that can save the encoded point cloud data generated by the source device 10, so that the destination device 20 may stream or download the stored data of the storage device 40 Coded point cloud data.
  • Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors.
  • the memory may include random access memory (RAM), read-only memory ( read-only memory, ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, which can be used to store the desired program in the form of instructions or data structures that can be accessed by a computer Any other media of the code, etc.
  • both the source device 10 and the destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.
  • the link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20.
  • the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded point cloud data to the destination device 20 in real time.
  • the source device 10 may modulate the coded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may send the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and/or wired communication media.
  • the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, and the packet-based network may be a local area network, a wide area network, or a global network (for example, the Internet).
  • the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application does not specifically limit this.
  • the storage device 40 may store the received encoded point cloud data sent by the source device 10, and the destination device 20 may directly obtain the encoded point cloud data from the storage device 40 .
  • the storage device 40 may include any of a variety of distributed or locally accessed data storage media.
  • any of the multiple distributed or locally accessed data storage media may be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or used to store Any other suitable digital storage media for encoding point cloud data, etc.
  • the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, and the destination device 20 can be stored via streaming or downloading.
  • the file server may be any type of server capable of storing the coded point cloud data and transmitting the coded point cloud data to the destination device 20.
  • the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive.
  • FTP file transfer protocol
  • NAS network attached storage
  • the destination device 20 can obtain the coded point cloud data through any standard data connection (including an Internet connection).
  • Any standard data connection can include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, digital subscriber line (DSL), cable modem, etc.), or is suitable for obtaining the encoded data stored on the file server The combination of the two point cloud data.
  • the transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.
  • the point cloud decoding system shown in FIG. 1 is only one possible implementation, and the technology of the present application can not only be applied to the source device 10 that can encode the point cloud shown in FIG.
  • the destination device 20 for decoding point cloud data may also be applicable to other devices that can encode point clouds and decode encoded point cloud data, which is not specifically limited in the embodiment of the present application.
  • the source device 10 includes a data source 120, an encoder 100 and an output interface 140.
  • the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, where the transmitter may also be referred to as a transmitter.
  • the data source 120 may include a point cloud capture device (for example, a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or A computer graphics system used to generate point cloud data, or a combination of these sources of point cloud data.
  • the data source may send a point cloud to the encoder 100, and the encoder 100 may encode the point cloud received from the data source 120 to obtain encoded point cloud data.
  • the encoder can send the encoded point cloud data to the output interface.
  • the source device 10 directly sends the encoded point cloud data to the destination device 20 via the output interface 140.
  • the encoded point cloud data may also be stored on the storage device 40 for the destination device 20 to obtain later and use for decoding and/or playback.
  • the destination device 20 includes an input interface 240, a decoder 200 and a display device 220.
  • the input interface 240 includes a receiver and/or a modem.
  • the input interface 240 can receive the encoded point cloud data via the link 30 and/or from the storage device 40, and then send it to the decoder 200.
  • the decoder 200 can decode the received encoded point cloud data to obtain the Decoded point cloud data.
  • the decoder may send the decoded point cloud data to the display device 220.
  • the display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data.
  • the display device 220 may be any one of multiple types of display devices.
  • the display device 220 may be a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED). Display or other type of display device.
  • LCD liquid crystal display
  • OLED organic light-emitting
  • the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software, used for encoding both audio and video in a common data stream or separate data streams.
  • MUX-DEMUX multiplexer-demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
  • Each of the encoder 100 and the decoder 200 can be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuit, ASIC ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors.
  • Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or the decoders may be integrated as a combined encoding in the corresponding device Part of the decoder/decoder (codec).
  • This application may generally refer to the encoder 100 as “signaling” or “sending” certain information to another device such as the decoder 200.
  • the term “signaling” or “sending” may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. retrieve the syntax element at any time.
  • FIG. 2 is a schematic block diagram of an encoder 100 provided by an embodiment of the application.
  • Fig. 2 illustrates the point cloud compression (PCC) coding framework of MPEG (moving pictrue expert group) as an example.
  • the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, image-based or video-based Encoding module 107, occupancy map encoding module 108, auxiliary information encoding module 109, multiplexing module 110, point cloud occupancy map down-sampling module 111, point cloud occupancy map filling module 112, point cloud reconstruction module 113, and point cloud filtering module 114 .
  • PCC point cloud compression
  • the point cloud block information generation module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds.
  • each point cloud group may include one or more frames of point clouds.
  • the current point cloud group will be unified in the following Take the current point cloud frame in as an example.
  • the point cloud block information generation module 101 can determine the three-dimensional coordinates of each point included in the current point cloud frame in the three-dimensional space coordinate system, and the normal direction vector of each point in the three-dimensional space, and determine each point according to the determined
  • the normal direction vector in the three-dimensional space and the predefined projection plane divide the current point cloud frame into multiple patches (point cloud blocks). Each patch includes one or more points in the current point cloud frame.
  • the point cloud block information generating module 101 can also determine the occupancy map of each patch and the depth map of each patch.
  • the point cloud block information generating module 101 can also determine the three-dimensional information of each patch and the normal coordinate axis information of each patch, which can be used as auxiliary information of each patch.
  • the point cloud block information generating module 101 can divide the patch into three types, namely the first type patch, the second type patch, and the third type patch.
  • the first identifier is used to indicate the type of each patch.
  • the point cloud block information generating module 101 may send the first identification of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be called compression encoding.
  • the three-dimensional information of each patch may include the offset of the patch to be encoded in the 3D space along the tangent direction, the offset of the patch to be encoded in the 3D space along the double tangent direction, and the offset of the patch to be encoded in the 3D space. Offset in the normal direction.
  • the point cloud block information generating module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102.
  • the point cloud block information generating module 101 can also send the depth map of each patch to the depth map generating module 103.
  • the packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101 to obtain the occupancy map of the current point cloud frame.
  • the packaging module 102 can arrange the occupancy map of each patch in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch, and then arrange the occupancy map of each patch in descending order (or ascending order). In the order of the occupancy map, insert the occupancy map of each patch into the available area of the occupancy map of the current point cloud frame to obtain the occupancy map of the current point cloud frame and the packing information of the patch.
  • the packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding.
  • the packing information of the patch may include the two-dimensional information of each patch in the occupancy map of the current point cloud frame, the identification information of the reference patch of each patch, the rotation transformation information of each patch, and the reference of the current point cloud frame The identification information of the frame.
  • the two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch.
  • the two-dimensional information of each patch can include the offset of each patch along the u axis in the current point cloud frame occupancy map, the offset of each patch along the v axis in the current point cloud frame occupancy map, and each patch
  • the current point cloud frame occupies the width of the figure, and each patch occupies the height of the figure in the current point cloud frame.
  • the packing module 102 may also send the packing information of the patch to the depth map generating module 103 and the point cloud occupancy map down-sampling module 111.
  • FIG. 4 is a schematic diagram of the patch of the point cloud of the frame
  • Fig. 5 is the occupancy diagram of each patch of the point cloud of the frame shown in Fig. 4, the frame point obtained by packing by the packing module 102 Schematic diagram of cloud occupancy map.
  • the point cloud shown in FIG. 3 may be the current point cloud frame in the embodiment of the present application
  • the patch of the point cloud shown in FIG. 4 may be the patch of the current point cloud frame in the embodiment of the present application
  • the cloud occupancy map may be the occupancy map of the current point cloud frame in the embodiment of the present application.
  • the auxiliary information encoding module 109 receives the occupancy map of the current point cloud frame and the packing information of the patch sent by the packing module 102, as well as the first identification of each patch and the first identification of each patch sent by the point cloud block information generating module 101. After the auxiliary information, the current point cloud frame and each patch included in the current point cloud frame can be encoded to obtain a code stream including the encoded auxiliary information, and then the obtained code stream including the encoded auxiliary information is sent to the complex Use module 110. Among them, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the packaging information of the patch may also be referred to as auxiliary information of each patch.
  • the depth map generation module 103 may receive the patch packaging information sent by the packaging module 102 and the depth map of each patch sent by the point cloud block information generation module 101, and then according to the packaging information of each patch and each The depth map of the patch generates the depth map of the current point cloud frame. Then the generated depth map of the current point cloud frame is sent to the depth map filling module 105 to fill the blank pixels in the depth map of the current point cloud frame to obtain the filled depth map of the current point cloud frame.
  • the depth map filling module 105 may send the obtained filled depth map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled depth map of the current point cloud frame, Obtain the reconstructed current point cloud depth map and the code stream including the encoded current point cloud frame depth map, and may send the obtained reconstructed current point cloud frame depth map to the point cloud reconstruction module 113 , And send the code stream including the encoded depth map of the current point cloud frame to the multiplexing module 110.
  • the point cloud occupancy map down-sampling module 111 may perform down-sampling processing on the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain a low-resolution occupancy map of the current point cloud frame. After that, the point cloud occupancy map down-sampling module 111 may also send the occupancy map of the current low-resolution point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112.
  • the occupancy map encoding module 108 can encode the received occupancy map of the current low-resolution point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 can also The code stream including the coded low-resolution occupancy map of the current point cloud frame may be sent to the multiplexing module 110.
  • the point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the low-resolution current point cloud frame to obtain the filled occupancy map of the current point cloud frame, The occupancy map of the filled current point cloud frame has the original resolution.
  • the point cloud reconstruction module 113 may be based on the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, and the reconstructed current point cloud frame sent by the image or video-based encoding module 107.
  • the depth map, patch packing information and patch auxiliary information reconstruct the geometry of the current point cloud frame to output the reconstructed point cloud.
  • the point cloud reconstruction module 113 can also output the reconstructed points in the reconstructed point cloud. Correspondence with patch, and the packaging position of the reconstructed point in the reconstructed point cloud.
  • the point cloud reconstruction module 113 can send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 can also reconstruct the reconstructed point cloud.
  • the packing positions of the points are sent to the texture generation module 104.
  • the point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be referred to as a smooth reconstructed point cloud. Or it can be said that the point cloud filtering block 114 can perform smoothing processing on the reconstructed point cloud.
  • the texture map generating module 104 receives the smooth reconstructed point cloud sent by the point cloud filtering module 114, the packed position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstruction module 113, and is sent by the data source 120 After the current point cloud frame, the texture map of the current point cloud frame can be generated according to the smooth reconstructed point cloud, the packaging position of the reconstructed point in the reconstructed point cloud and the current point cloud frame, and the generated current point cloud The texture map of the frame is sent to the texture map filling module 106 to fill the blank pixels in the texture map of the current point cloud frame to obtain the filled texture map of the current point cloud frame.
  • the texture map filling module 106 may send the obtained filled texture map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled texture map of the current point cloud frame, Obtain the code stream including the texture map of the reconstructed current point cloud frame.
  • the image or video-based encoding module 107 may also send the obtained code stream including the reconstructed texture map of the current point cloud frame to the multiplexing module 110.
  • the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 can send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 can combine the received code streams To form a combined code stream, and send the combined code stream to the output interface 140.
  • the output interface 140 may send the combined code stream to the decoder 200.
  • the encoder 100 shown in FIG. 2 is only an embodiment provided by the present application.
  • the encoder 100 may include more or less modules than those shown in FIG. 2 Module.
  • the embodiments of the present application do not specifically limit this.
  • FIG. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the application.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, and point cloud geometric information reconstruction Structuring module 206, point cloud filtering module 207, and point cloud texture information reconstruction module 208.
  • the demultiplexing module 201 may receive the combined code stream sent by the output interface 140 of the encoder 100 through the input interface 204, and send the combined code stream to the corresponding decoding module. Specifically, the demultiplexing module 201 sends the code stream of the texture map of the current point cloud frame encoded and the code stream of the depth map of the encoded current point cloud frame to the image or video-based decoding module 202, and The code stream including the occupancy map of the current point cloud frame with the encoded low resolution is sent to the occupancy map decoding module 203, and the code stream including the encoded auxiliary information is sent to the auxiliary information decoding module 204.
  • the image or video-based decoding module 202 can decode the received bitstream including the texture map of the encoded current point cloud frame and the bitstream including the depth map of the encoded current point cloud frame to obtain the reconstructed current
  • the texture map information of the point cloud frame and the depth map information of the reconstructed current point cloud frame, and the reconstructed texture map information of the current point cloud frame can be sent to the point cloud texture information reconstruction module 208,
  • the constructed depth map information of the current point cloud frame is sent to the point cloud geometric information reconstruction module 206.
  • the occupancy map decoding module 203 can decode the received bitstream including the occupancy map of the current point cloud frame with encoded low resolution to obtain the occupancy map information of the reconstructed current point cloud frame with low resolution, and
  • the reconstructed low-resolution current point cloud frame occupancy map information is sent to the point cloud occupancy map filling module 205.
  • the point cloud occupancy map filling module 205 can obtain the reconstructed occupancy map information of the current point cloud frame with the original resolution according to the occupancy map information of the reconstructed low-resolution current point cloud frame, and then the reconstructed has The occupancy map information of the current point cloud frame at the original resolution is sent to the geometric information reconstruction module 206 of the point cloud.
  • the reconstructed occupancy map information of the current point cloud frame with the original resolution will be uniformly referred to as the occupancy map information of the reconstructed current point cloud frame.
  • the auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the geometric information reconstruction module 206 of the point cloud.
  • the specific decoding method will be described in subsequent embodiments.
  • the geometric information reconstruction module 206 of the point cloud can be based on the received depth map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, and the reconstructed information sent by the point cloud occupancy map filling module 205.
  • the occupancy map information of the current point cloud frame and the auxiliary information sent by the auxiliary information decoding module 204 reconstruct the geometry of the current point cloud frame to obtain the reconstructed point cloud.
  • the reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process can refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100. I won't repeat it here.
  • the geometric information reconstruction module 206 of the point cloud may also send the reconstructed point cloud to the point cloud filtering module 207.
  • the point cloud filtering module 207 can filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud.
  • the specific filtering process can refer to the filtering process of the point cloud filtering module 114 in the encoder 100 , I won’t repeat it here.
  • the point cloud filtering module 207 may send the smooth reconstructed point cloud to the texture information reconstruction module 208 of the point cloud.
  • the texture information reconstruction module 208 of the point cloud receives the smooth reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202. Afterwards, the texture information of the reconstructed point cloud can be reconstructed to obtain the reconstructed point cloud reconstructed by the texture information.
  • the decoder 200 shown in FIG. 6 is only an example, and in a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 6. The embodiment of the present application does not limit this.
  • the current point cloud group may be a group of frame (GOF), and a frame group may include 32 frames of point clouds, that is, the current point cloud group may include 32 frames of point clouds.
  • the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application.
  • a frame group may also be divided into multiple subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the multiple subgroups.
  • FIG. 7 is a schematic diagram of a frame group provided by an embodiment of the application. As shown in FIG.
  • a frame group can be divided into 10 subgroups, and the first subgroup includes the first, second, and third subgroups.
  • Frame point cloud the second subgroup includes the 4th frame point cloud
  • the third subgroup includes the 5th, 6, 7, and 8th frame point clouds
  • the 10th subgroup includes the 30th, 31st, and 32th frame point clouds
  • the current point cloud group can be any one of these 10 subgroups.
  • FIG. 7 only exemplarily shows the point clouds included in the first subgroup, the second subgroup, the third subgroup, and the tenth subgroup, and the point clouds included in the other subgroups are not described in detail.
  • the current point cloud group can be a frame group, or the current point cloud group can be a subgroup of a frame group.
  • the current point cloud group can also be other forms of point cloud groups, such as custom ones.
  • the point cloud group is not limited in this embodiment of the application.
  • patches can be divided into three types, namely the first type of patch, the second type of patch and the third type of patch.
  • the patch can also be divided into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, the first type of patch can be a global matching patch, the second type of patch can be a local matching patch, and the third type of patch can be a non-matching patch.
  • the three types of patches are described.
  • the patch to be coded is any patch in the current point cloud frame, and the current point cloud frame is any of the current point cloud groups.
  • a frame of point cloud If the patch to be coded is the first type of patch, the patch to be coded has a matching relationship in all the point cloud frames in the current point cloud group, this can be called a global matching patch; if the patch to be coded is the second For the patch type, the patch to be encoded has a patch with a matching relationship in the reference frame of the current point cloud frame, but only a patch with a matching relationship exists in a part of the point cloud frame in the current point cloud group.
  • the reference frame of the current point cloud frame may be located in the current point cloud group.
  • the reference frame of the current point cloud frame may be the previous point cloud frame of the current point cloud frame.
  • the patches included in all point cloud frames in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch.
  • the first type of patch in the current point cloud group is The number in each point cloud frame is the same, and when the number of the first-type patches is greater than 1, the first-type patches in each point cloud frame in the current point cloud group can also be arranged in the same order , That is, there is a matching relationship between the first-type patches at the same position in each point cloud frame.
  • the w-th first-type patch in each point cloud frame has a matching relationship with each other, and w is less than or equal to the number of first-type patches.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the first-type patch in the occupancy map of the current point cloud frame are the same, but the three-dimensional information is different.
  • the two-dimensional information of the first-type patch can also be different, or the three-dimensional information can also be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. In this way, for two-dimensional information and three-dimensional information, there may be three situations.
  • the following takes the patch to be coded as an example.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded Same as the three-dimensional information of the patch to be encoded.
  • the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded
  • the three-dimensional information of the reference patch of the patch to be encoded is the same as The three-dimensional information of the patch to be encoded is different.
  • the two-dimensional information of the reference patch of the patch to be encoded is different from the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded The information is different from the three-dimensional information of the patch to be encoded.
  • the reference patch of the patch to be coded may be the same position in the reference frame of the current point cloud frame where the patch to be coded is located Patch. For example, if the patch to be coded is in the third position in the current point cloud frame, the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be coded.
  • the rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information.
  • the rotation angle information includes a specific rotation angle value or rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch to be coded is rotated by 20°; if the rotation angle of the patch to be coded is A value of 80 means that the patch to be coded is rotated by 80°.
  • the rotation angle index of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch to be coded is rotated by 15°; if the rotation angle of the patch to be coded is If the index is 2, it means that the patch to be coded is rotated by 30°; if the rotation angle index of the patch to be coded is 3, it means that the patch to be coded is rotated by 45°.
  • the rotation angle information can also be characterized in other forms, which is not specifically limited here. For example, it can also be described in the form of a matrix, especially when accelerated processing is performed in a GPU (graphic process unit, image processor), the use of the matrix form will increase the processing speed.
  • rotation of the patch to be encoded can be clockwise or counterclockwise.
  • the coordinate axis exchange information can be expressed as the coordinate axis exchange flag.
  • the coordinate axis exchange flag when the coordinate axis exchange flag is false or 0, it indicates the tangent coordinate axis of the patch to be encoded and the double tangent coordinate of the patch to be encoded
  • the axes are not swapped; for example, when the coordinate axis swap flag is ture or 1, the tangent coordinate axis of the patch to be coded and the double tangent coordinate axis of the patch to be coded are swapped.
  • the index of the coordinate axis of the patch to be encoded can be 0, 1, or 2.
  • the tangent axis index of the patch to be encoded and the double tangent axis index of the patch to be encoded can be 0, 1, and 2. Any two.
  • the tangent axis of the patch can be called the U axis
  • the double tangent axis of the patch can be called the V axis.
  • the obtained tangent axis index of the current patch and the double tangent axis index of the patch to be encoded are 0 and 2, respectively.
  • the tangent of the patch to be encoded is not exchanged
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100.
  • FIG. 9 is a flowchart of a point cloud encoding method provided by an embodiment of the present application, and the method is applied to a point cloud decoding system. Referring to Figure 9, the method includes:
  • S901 Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.
  • the patch to be encoded may be any patch included in the current point cloud frame to be encoded.
  • the first identifier may indicate the type of patch to be encoded.
  • the first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.
  • the first identifier is the first value
  • the first type of patch can be a global matching patch in the current point cloud group.
  • the auxiliary information of the patch to be encoded includes three-dimensional information. Including two-dimensional information, rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be coded may include three-dimensional information, but not two-dimensional information, rotation transformation information and normal direction. Coordinate axis information.
  • the patches in each point cloud frame can be arranged in the order of the first type of patch, the second type of patch and the third type of patch, and then the first type of patch and the second type of patch are also used when encoding.
  • the type patch and the third type patch are coded sequentially. So if there is a first-type patch in the current point cloud group, the first patch in each point cloud frame is the first-type patch.
  • the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first-type patches is greater than 1, the number of the first type of patches in each point cloud frame in the current point cloud group
  • One type of patches can be arranged in the same order, that is, there is a matching relationship between the first type of patches at the same position in each point cloud frame. If the first-type patch does not exist in the current point cloud group, the first patch in each point cloud frame may be the second-type patch or the third-type patch. If the first patch in each point cloud frame is a third-type patch, it can indicate that only this point cloud frame is included in the current point cloud group, and the first identifiers of all patches in this point cloud frame are third value.
  • the first identifier of all patches included in the first point cloud frame can be set to the third value, and the patches in other point cloud frames are still in accordance with the first type of patch and the second type.
  • the patch and the third type of patch are arranged in the order. Therefore, when the first identifier of the patch to be encoded is the first value, it can be determined that the current point cloud frame is the non-first point cloud frame in the current point cloud group.
  • the current point cloud frame may be the Kth point cloud frame in the current point cloud group, and K is an integer greater than or equal to 3. That is, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the manner of S902.
  • the point cloud frames in the current point cloud group can be numbered according to two numbering methods.
  • the first numbering method point cloud frame 0, point cloud frame 1, point cloud frame 2...;
  • the second numbering method point cloud frame 1, point cloud frame 2, point cloud frame 3... Therefore, the above Kth point cloud frame can refer to the point cloud frame 2 in the first numbering method and any point cloud frame after the point cloud frame 2, or it can refer to the point cloud frame 3 and the point cloud frame in the second numbering method.
  • the Kth point cloud frame can refer to the third point cloud frame in the actual current point cloud group and the point cloud frames after the third.
  • the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be encoded in the manner of S902, and for the second point cloud frame in the current point cloud group
  • a point cloud frame can be encoded in the following two ways:
  • the first possible implementation manner if the current point cloud frame is the second point cloud frame in the current point cloud group, then, when the first identifier of the patch to be encoded is the first value, and the patch to be encoded is the current point cloud
  • the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded can be incorporated into the code stream.
  • the auxiliary information of the patch to be coded can be encoded into the code stream instead of the patch to be coded
  • the first identifier is incorporated into the code stream.
  • the number of first-type patches in the current point cloud group can be obtained.
  • the operation of encoding the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: editing the first identifier of the patch to be encoded Enter the code stream, and then encode the number of first-type patches in the current point cloud group into the code stream, and finally, encode the auxiliary information of the patch to be encoded into the code stream. That is, the number of the first-type patch in the current point cloud group is compiled into the code stream after the first identifier of the patch to be encoded.
  • the fixed-length encoding method can be used to encode the number of patches of the first type into the code stream, and the variable-length encoding method can also be used to encode the number of patches of the first type into the code stream. Not limited.
  • the current point cloud frame is the second point cloud frame in the current point cloud group
  • the first value of the patch to be coded can be set
  • An identification and auxiliary information are compiled into the code stream. That is, the first identification and auxiliary information of all the first-type patches in the second point cloud frame need to be encoded into the code stream, so that the number of the first-type patches in the current point cloud group can be indicated without The number of the first type patch in the current point cloud group is compiled into the code stream.
  • the number of the first-type patch in the current point cloud group can be determined through the second point cloud frame.
  • the first patch that is, the first identifier of the first first type patch can be coded It is not necessary to encode the first identifier of the non-first first-type patch into the code stream.
  • the first identifier of the patch to be encoded is the third value
  • the patch to be encoded is the first patch in the current point cloud frame
  • the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame only the first identifier of the first type patch can be compiled into the code stream, instead of the non-first point cloud frame.
  • the first identifier of the first type patch is encoded into the code stream, which improves the coding efficiency and reduces the bit overhead of the code stream.
  • the first patch in the current point cloud frame can correspond to an identification information, which can also be called an index, and this identification information can uniquely indicate the first patch in the current point cloud frame patch, and the identification information of the first patch can usually be incorporated into the code stream.
  • an identification information which can also be called an index
  • this identification information can uniquely indicate the first patch in the current point cloud frame patch
  • the identification information of the first patch can usually be incorporated into the code stream.
  • a counter can be used to determine which patch is the current point cloud frame. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).
  • the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded.
  • the 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited.
  • the fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.
  • differential encoding to encode the three-dimensional information of the patch to be encoded into the code stream refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, as shown in Table 1 and Table 2.
  • skip_patch_data_unit in Table 1 is the information of the patch to be encoded that is encoded into the code stream when the first identifier of the patch to be encoded is the first value, which can also be called the data unit of the patch to be encoded.
  • the auxiliary information of the patch to be encoded and the first identifier can be encoded into the code stream. That is, for each patch in the first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch can be encoded into the code stream. For the non-first-type patches in the non-first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch can be encoded into the code stream.
  • the first identifier when the first identifier is the second value, it indicates that the patch to be encoded is a second type of patch.
  • the second type of patch can be a local matching patch in the current point cloud group.
  • the auxiliary information of the patch to be encoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be encoded.
  • the first identifier is the third value, it indicates that the patch to be coded is a third type of patch.
  • the third type of patch can be a non-matching patch in the current point cloud group.
  • the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, Rotation transformation information and normal coordinate axis information.
  • the second type of patch has a matching patch in the reference frame of the current point cloud frame, and the rotation transformation information and the normal coordinate axis information between the second type of patch and its reference patch are the same, while the two-dimensional information It may be different from the three-dimensional information.
  • the third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first When the value is one, the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the two-dimensional information of the patch to be encoded may include the offset (2d_shift_u) of the patch to be encoded in the current point cloud frame occupancy map along the u axis, and the patch to be encoded in the current point cloud frame occupancy map along the v axis
  • the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex.
  • the syntax element of the rotation transformation information of the patch to be encoded may be rotation.
  • the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded.
  • the normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.
  • the two-dimensional information and three-dimensional information of the patch to be encoded can be compiled into the code stream according to the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded.
  • the differential encoding method encodes the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream, and a more accurate predictive differential method can also be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream.
  • the specific encoding method is not used. limited.
  • the identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method.
  • the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal axis information and the first identifier of the patch to be encoded into the code stream, or variable
  • the long encoding method encodes the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and first identifier of the patch to be encoded into the code stream, which is not limited in the embodiment of the present application.
  • using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be encoded, and The difference between the three-dimensional information of the coded patch and the three-dimensional information of the reference patch of the patch to be coded is encoded into the code stream.
  • the first identifier of the first first-type patch in the current point cloud frame the first identifier of each second-type patch, and the first identifier of each third-type patch in the current point cloud frame
  • the logo needs to be programmed into the code stream.
  • the auxiliary information of each patch included in the current point cloud frame also needs to be compiled into the code stream. There may be the following three possible implementation methods for the encoding method of the first identification and auxiliary information.
  • the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches.
  • the auxiliary information of each patch included in the current point cloud frame can be individually packaged and compiled into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Then, the auxiliary information of each patch included in the current point cloud frame is packaged one by one and compiled into the code stream.
  • the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately.
  • Code stream, the auxiliary information of each patch included in the current point cloud frame can also be packaged separately and compiled into code stream.
  • the first identifier of the first patch in the current point cloud frame can be compiled into the code stream, and then the auxiliary information of the first patch can be compiled into the code stream. Then, the auxiliary information of other first-type patches in the current point cloud frame is compiled into the code stream one by one.
  • the coding can be performed in a way that the first identifier is first compiled, and then the auxiliary information is compiled.
  • the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches.
  • the same type of information in each patch auxiliary information can be packaged together and compiled into the code stream. Different types of information in the auxiliary information of each patch can be individually packaged and compiled into the code stream.
  • the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream.
  • the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information
  • some auxiliary information of patches includes two-dimensional information or three-dimensional information
  • some auxiliary information of patches does not include two-dimensional information or
  • the rotation transformation information and normal coordinate axis information in the auxiliary information of each patch are packaged together and compiled into the code stream.
  • the two-dimensional information and three-dimensional information in the auxiliary information of each patch are individually packaged and compiled into the code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.
  • the third type of patch when encoding is performed in the order shown in Figure 8, for the third type of patch, other methods other than the encoding methods described above can also be used for encoding, for example, if The patch to be coded is the first third-type patch, then the fixed-length coding method may be used for coding, or the variable-length coding method may be used, which is not limited in the embodiment of the application. If the patch to be coded is not the first type 3 patch, at this time, according to the 2D information and 3D information of the first type 3 patch, the 2D information and 3D information of the patch to be coded can be encoded into the code stream using differential encoding. , And other information is also encoded in the manner described above.
  • using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream means: the two-dimensional information and the first
  • the difference between the two-dimensional information of the third-type patch and the difference between the three-dimensional information of the patch to be encoded and the first three-dimensional patch of the third type are encoded into the code stream.
  • the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame.
  • the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting
  • the first identifier of the coded patch is encoded into the code stream, so that the format of the code stream can be simplified, the bit overhead of the code stream is further reduced, and the coding efficiency is improved.
  • any of the following point cloud decoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • the to be decoded can be parsed from the code stream The first ID of the patch.
  • the code stream includes the identification information of the patch to be decoded, and if the identification information of the patch to be decoded is included, it can be determined that the patch to be decoded is the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, it can be determined that the patch to be decoded is the non-first patch in the current point cloud frame. At this time, the number of the patch to be decoded in the current point cloud frame can be determined according to the counter. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point cloud frames in the current point cloud group.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the number of patches of the first type can be parsed from the code stream by a method that matches the encoding end.
  • fixed-length decoding or variable-length decoding can be used to parse the number of patches of the first type from the code stream.
  • the first identifier when the first identifier is the first value, it can indicate that the current point cloud frame is the non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the second point cloud frame in the current point cloud group, the patch to be decoded can be decoded through the following two possible implementation methods.
  • the first possible implementation is the first identification of the first patch in the second point cloud frame in the current point cloud group, and the number of the first type of patch included in each point cloud frame in the current point cloud group Code stream together.
  • the current point cloud frame is the second point cloud frame in the current point cloud group
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first patch to be decoded can be parsed from the code stream. Identification, when the first identification is the first value, the number of the first-type patches in the current point cloud group is analyzed from the code stream. Then parse the auxiliary information of the patch to be decoded from the code stream.
  • the first identifiers of the first type patch included in the second point cloud frame in the current point cloud group are all encoded into the code stream.
  • the first identifier of the patch to be decoded can be parsed from the code stream.
  • the first identifier is the first value
  • the patch to be decoded is parsed from the code stream. Supplementary information.
  • the first identifier of the non-first first-type patch is not encoded into the code stream, so when the patch to be decoded is the first in the current point cloud frame n patches, when n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first first-type patch in the current point cloud frame,
  • the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information
  • the patch to be decoded is the first type of patch
  • the auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.
  • a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream.
  • the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, and then according to the The identification information of the reference frame determines the reference frame of the current point cloud frame, and then obtains the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame, and refers to the patch
  • the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch.
  • the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame.
  • the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame
  • the auxiliary information of the patch is determined.
  • the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc.
  • the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can be obtained from the decoded point cloud, and the obtained two-dimensional information,
  • the rotation transformation information and the normal coordinate axis information are used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.
  • the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded at the current point Patches with the same position in the cloud frame.
  • the method of using differential decoding to parse the 3D information of the patch to be decoded from the code stream may specifically be: obtaining the 3D information of the patch whose position is the same as the position of the patch to be decoded in the current point cloud frame from the decoded point cloud. Or, obtain the three-dimensional information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame. Then, the obtained three-dimensional information is added with the difference of the three-dimensional information parsed from the code stream to obtain the three-dimensional information of the patch to be decoded.
  • the counter can be used to determine the number of patches in the current point cloud frame to be decoded.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame
  • the patch to be decoded can be determined It is the non-first first-type patch in the current point cloud frame, so that the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the number of patches of the first type included in each point cloud frame in the current point cloud group can be determined in two ways.
  • the first identifier in the current point cloud group can be directly parsed from the code stream. The number of class patches.
  • the number of patches with the first identifier as the first value can be determined, so as to determine The number of first-type patches included in the second point cloud frame, that is, the number of first-type patches in the current point cloud group. That is, the number of first-type patches included in each point cloud frame in the current point cloud group can be determined through the second point cloud frame.
  • the second point cloud frame in the current point cloud group can be used as the reference point cloud frame of the third point cloud frame
  • the third point cloud frame can be used as the reference point cloud frame of the fourth point cloud frame Point cloud frame, and so on.
  • the third point cloud frame and the point cloud frames after the third can determine whether the nth patch in the current point cloud frame is the first type patch according to the number of the first type patch in the reference point cloud frame.
  • the first identifier of the first patch in the current point cloud frame, and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately and compiled into the code stream, Can be packaged together and compiled into code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for patches that take the second value or the third value other than the first patch, the first identifier of the patch to be decoded can also be parsed from the code stream.
  • the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be parsed in sequence.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream
  • the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packaged and compiled into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, you can The code stream parses the same type of information in the auxiliary information of each patch included in the current point cloud frame. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • S1104 When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first-type patch in the reference point cloud frame of the current point cloud frame, analyze the to-be-decoded patch from the code stream The first ID of the patch.
  • the first identifier is encoded into the code stream, so when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current
  • the point cloud frame refers to the number of the first-type patch in the point cloud frame, it indicates that the patch to be decoded is a second-type patch or a third-type patch.
  • the first identifier of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information and three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.
  • the auxiliary information of the second type of patch that is encoded into the code stream during the encoding process includes two-dimensional information, three-dimensional information and identification information of the reference patch, but does not include the rotation transformation information and the normal coordinate axis information, so when the patch to be decoded When it is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
  • the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis.
  • the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained.
  • the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner.
  • the reference patch of the patch to be decoded can be determined by the identification information of the reference patch of the patch to be decoded, and then the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded can be obtained. Then, the obtained two-dimensional information and three-dimensional information are respectively added with the two-dimensional information difference and the three-dimensional information difference parsed from the code stream, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.
  • the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively.
  • the application embodiment does not limit this.
  • the first type patch, the second type patch and the third type patch in each point cloud frame in the current point cloud group can be arranged as shown in Figure 8.
  • the patch to be decoded is The first patch in the current point cloud frame
  • the fixed-length decoding method can be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal axis information of the patch to be decoded from the code stream, or the variable-length decoding method can be used to parse the to-be-decoded patch from the code stream.
  • the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the decoded patch are not limited in the embodiment of the present application.
  • the auxiliary information of the first third-type patch is coded according to fixed-length coding or variable-length coding
  • the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by differential coding
  • other auxiliary information is coded Encode with fixed-length encoding.
  • differential decoding can be used to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream
  • fixed-length decoding can be used to parse the to-be-decoded patch from the code stream. Decode the rotation transformation information and normal axis information of the patch.
  • the method of using differential decoding to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream is: parse the two-dimensional information and the first information of the patch to be decoded from the code stream.
  • the difference between the two-dimensional information of the third-type patch is added to the two-dimensional information of the first third-type patch to obtain the two-dimensional information of the patch to be decoded.
  • the auxiliary information of the patch to be decoded that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded
  • the auxiliary information can be reproduced according to but not limited to these auxiliary information.
  • this application does not limit the specific information needed to reconstruct the patch to be decoded.
  • the first identifier of the first patch in the current point cloud frame is encoded into the code stream
  • the first identifier of the patch to be decoded is parsed from the code stream.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the information includes the three-dimensional information of the patch to be decoded.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information and information of the patch to be decoded.
  • the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, and Rotation transformation information and normal coordinate axis information.
  • the first identifier needs to be parsed from the code stream, while for the non-first type first patch, the first identifier does not need to be parsed from the code stream, and for the first type patch and the second type
  • the first identifier does not need to be parsed from the code stream, and for the first type patch and the second type
  • auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. In this way, the format of the parsed code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100. Referring to Figure 12, the method includes:
  • S1201 Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.
  • S1201 may refer to S901 in the embodiment shown in FIG. 9, and this is not repeated in this embodiment of the application.
  • the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and information between the first-type patches in the current point cloud group
  • the normal coordinate axis information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch may also be the same.
  • the first identifier is the first value, it may indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream may also include the second identifier.
  • the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the syntax element of the second identification may be override_3d_shift_data_flag.
  • the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information.
  • the second identifier is the fifth value
  • the specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used.
  • the syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.
  • this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier, see Table 1 and Table 3.
  • the information encoded into the code stream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the third-identified syntax element may be override_2d_shift_data_flag. Among them, the second identifier has already been described, so it will not be repeated here.
  • the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded can be Does not include two-dimensional information.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the two-dimensional information of the patch to be encoded is not the same as the two-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include Two-dimensional information.
  • the syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.
  • this article illustrates a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier and the third identifier, see Table 1 and Table 4 .
  • the two-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, not only the above-mentioned second and third marks can be used to indicate the coding Whether the information of the stream includes two-dimensional information and three-dimensional information of the patch to be encoded. It can also be indicated in other ways.
  • the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream, that is, when the first identifier is the first value , It can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include a fourth identifier.
  • the fourth identified syntax element may be override_patch_data_flag.
  • the fourth identifier is the eighth value, it may indicate that the two-dimensional information and three-dimensional information of the patch to be encoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded, and the code stream is not included.
  • the fourth identifier is a ninth value
  • it can indicate that the two-dimensional information and three-dimensional information of the patch to be encoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded, and both have been encoded into the code stream.
  • the syntax element of the eighth value can be flase, and the syntax element of the ninth value can be true.
  • the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), and The offset of the encoded patch in the 3D space along the normal direction (3d_shift_normal_axis).
  • the two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v).
  • each of the above identifiers can have two values, and the syntax elements of the two values can be true and false.
  • the meaning of each value can refer to the meaning of different values of the second identifier, the third identifier or the fourth identifier, which will not be repeated here.
  • this article exemplifies when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the fourth identifier, and the identifier used to indicate different two-dimensional information and three-dimensional information. Part of the grammatical structure, see Table 1 and Table 5.
  • the first identifier when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.
  • the first identifier when it is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the auxiliary information, the first identifier and the second identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, When the patch to be coded is the non-first patch in the current point cloud frame, it indicates that the patch to be coded is the non-first type 1 patch in the current point cloud frame, and the auxiliary information and the second identifier of the patch to be coded can be coded Instead of coding the first identifier of the patch to be coded into the code stream, the format of the code stream can be simplified.
  • the second identifier is an identifier used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream.
  • the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. Information, in this way, can further reduce the bit overhead of the code stream, thereby improving the coding efficiency.
  • any of the following point cloud decoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • the second identifier when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the 3D information of the patch to be decoded is the same as the 3D information of the reference patch of the patch to be decoded.
  • the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded. Specifically, the auxiliary information of the reference patch of the patch to be decoded may be used as the auxiliary information of the patch to be decoded.
  • the second identifier When the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information.
  • the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.
  • a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.
  • the information of the codestream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded can also be parsed from the code stream.
  • the third identifier is the sixth value, it can indicate that the two-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be decoded is the same as the two-dimensional information of the reference patch of the patch to be decoded, so the two-dimensional information of the reference patch of the patch to be decoded can be used as the two-dimensional information of the patch to be decoded.
  • the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be decoded has been encoded into the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • step 1202 when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are different, not only
  • the above-mentioned second identifier and third identifier can be used to indicate whether the information compiled into the code stream includes the two-dimensional information and three-dimensional information of the patch to be decoded. It can also be indicated in other ways.
  • the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be decoded have been encoded into the code stream.
  • the fourth identifier of the patch to be decoded can be parsed from the code stream.
  • the fourth identifier is the eighth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not encoded, so the to-be decoded
  • the 2D information and 3D information of the reference patch of the patch are used as the 2D information and 3D information of the patch to be decoded.
  • the fourth identifier is the ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. Parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream.
  • the specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.
  • the identifier used to indicate whether 2d_shift_u has been encoded into the code stream (override_2d_shift_u)
  • the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v)
  • the second identifiers of the non-first first-type patch in the current point cloud frame are all encoded into the bitstream, when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current
  • the point cloud frame refers to the number of the first-type patch in the point cloud frame, that is, when the patch to be decoded is not the first first-type patch
  • the second identifier of the patch to be decoded can be parsed from the code stream.
  • the first identifier of the first patch in the current point cloud frame is encoded into the code stream
  • the point cloud block patch to be decoded is the first patch in the current point cloud frame
  • the first identifier of the patch to be decoded is parsed in the code stream.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first patch of the first type.
  • the second identifier of the patch to be decoded can be parsed from the code stream.
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the second identifier of the patch to be decoded can be parsed from the code stream.
  • the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded.
  • the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.
  • any of the following point cloud encoding methods may be the point cloud decoding system.
  • the execution by the encoder 100 may be executed by the auxiliary information encoding module 109 in the encoder 100. Referring to Figure 14, the method includes:
  • S1401 Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.
  • the first identifier When the first identifier is the first value, it indicates that the patch to be encoded is the first type of patch.
  • the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and two-dimensional information between the first-type patches in the current point cloud group , 3D information, rotation transformation information and normal coordinate axis information are all the same. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is encoded into the code stream instead of the auxiliary information of the patch to be coded. Code stream.
  • the first identifier of the patch to be encoded is the first value, the second value or the third value
  • the first identifier needs to be encoded into the code stream.
  • the first identifiers of all patches in the current point cloud frame can be packaged into the code stream. Of course, they can also be packaged into the code stream separately.
  • Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, and for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.
  • any of the following point cloud decoding methods may be the point cloud decoding system.
  • What is executed by the decoder 200 more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200.
  • the decoding method includes:
  • the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially.
  • the patch to be decoded is the first patch in the current point cloud frame
  • the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.
  • a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream.
  • a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.
  • the auxiliary information of the patch to be decoded includes 2D information, 3D information, rotation transformation information, and method. Axis information.
  • the reference patch of the patch to be decoded can be Auxiliary information to obtain 2D information, 3D information, rotation transformation information and normal coordinate axis information of the patch to be decoded.
  • the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, and then from the reference frame of the current point cloud frame Obtain the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded, and use the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch as the information to be decoded Patch's two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch.
  • the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame.
  • the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame
  • the auxiliary information of the patch is determined.
  • the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc.
  • the position of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can also be obtained from the decoded point cloud.
  • Two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information, and the obtained two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information.
  • the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded at the current point Patches with the same position in the cloud frame.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded.
  • the auxiliary information of the second type of patch that is incorporated into the code stream during the encoding process includes two-dimensional information, three-dimensional information, and identification information of the reference patch, but does not include rotation transformation information and normal coordinate axis information
  • the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and information of the patch to be decoded.
  • Normal axis information At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the first identifier of the point cloud block patch to be decoded is first parsed from the code stream.
  • the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, and the code stream does not include the Decode the auxiliary information of the patch.
  • the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.
  • the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch.
  • the auxiliary information of the patch to be decoded can be parsed from the code stream.
  • the auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be decoded. When the first identifier is the third value, it indicates that the patch to be decoded is of the third type.
  • the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and rotation. Transformation information and normal axis information. That is, for the first type of patch, only the first identifier needs to be parsed from the code stream, and other auxiliary information can be derived or obtained.
  • auxiliary information For the second type of patch, only part of the auxiliary information needs to be parsed from the code stream. Other auxiliary information can also be derived or obtained. In this way, the format of the parsing code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.
  • the encoding apparatus 1600 may include:
  • the patch information obtaining module 1601 is used to obtain auxiliary information of the point cloud block to be coded and the first identifier of the patch to be coded;
  • the auxiliary information encoding module 1602 is used to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame; or
  • the auxiliary information encoding module 1602 is used to encode the auxiliary information of the patch to be encoded into the code stream when the first identifier is the first value, and the patch to be encoded is the non-first patch in the current point cloud frame;
  • the first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information.
  • the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.
  • the auxiliary information encoding module 1602 is also used for:
  • the first identifier is the second value or the third value
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the auxiliary information encoding module 1602 is also used for:
  • the first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches are excluding the patch to be encoded
  • the remaining first identifier besides the first identifier is the second value or the third value.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively,
  • the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  • the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.
  • the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.
  • the first type of patch is a global matching patch in the current group
  • the second type of patch is a local matching patch in the current group
  • the third type of patch is a non-matching patch in the current group.
  • the patch information acquisition module 1601 is used to execute the related content of step S901 of the embodiment shown in FIG. 9, and the auxiliary information encoding module 1602 is used to execute the related content of step S902 and step S903 of the embodiment shown in FIG. 9.
  • the patch information acquisition module 1601 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words,
  • the function of the patch information acquisition module 1601 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG.
  • the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in FIG. 2.
  • the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded
  • the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • FIG. 17 is a schematic structural diagram of the first point cloud decoding apparatus provided by an embodiment of the application.
  • the decoding apparatus 1700 may include:
  • the auxiliary information decoding module 1701 is used to parse the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame;
  • the auxiliary information decoding module 1701 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded;
  • the auxiliary information decoding module 1701 is also used for when the patch to be decoded is the nth patch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of the first-type patch in the reference point cloud frame of the current point cloud frame Analyze the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded;
  • the reconstruction module 1702 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.
  • the auxiliary information decoding module 1701 is also used for:
  • the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patch in the reference point cloud frame of the current point cloud frame, parse the patch to be decoded from the code stream First logo
  • the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded;
  • the auxiliary information of the patch to be decoded is parsed from the code stream.
  • the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.
  • the auxiliary information decoding module 1701 is also used for:
  • the first identifiers of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed include the patch to be decoded A first identifier, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value.
  • the first value, the second value and the third value are binary 10, 11 and 00 respectively,
  • the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.
  • the current point cloud frame is the first point cloud frame.
  • the auxiliary information decoding module 1701 is further configured to:
  • the reconstruction module 1702 is also used to:
  • the patch to be decoded is reconstructed.
  • the auxiliary information decoding module 1701 is further configured to:
  • the reconstruction module 1702 is also used to:
  • the patch to be decoded is reconstructed.
  • the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.
  • the auxiliary information decoding module 1701 is used to execute the related content of steps S1101-S1106 in the embodiment shown in FIG. 11, and the reconstruction module 1702 is used to execute the related content of step S1107 in the embodiment shown in FIG.
  • the auxiliary information decoding module 1701 in FIG. 17 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1702 in FIG. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in FIG. 6.
  • this application is not limited to this.
  • FIG. 18 is a schematic structural diagram of the first point cloud encoding device provided by an embodiment of the application.
  • the encoding device 1800 may include:
  • the patch information obtaining module 1801 is used to obtain auxiliary information of the point cloud block to be coded and the first identifier of the patch to be coded;
  • the auxiliary information encoding module 1802 is used to encode the auxiliary information of the patch to be encoded, the first identification, and the second identification of the patch to be encoded into the code stream when the first identifier is the first value, and the patch to be encoded is the current point cloud frame The first patch in; or
  • the auxiliary information encoding module 1802 is used to encode the auxiliary information of the patch to be encoded and the second identification of the patch to be encoded into the code stream when the first identifier is the first value.
  • the patch to be encoded is the non-first identifier in the current point cloud frame. Patch;
  • the first identifier is the first value to indicate that the patch to be encoded is the first type of patch
  • the second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream;
  • the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, and the auxiliary information of the patch to be encoded includes three-dimensional information.
  • the patch information acquisition module 1801 is used to execute the related content of step S1201 in the embodiment shown in FIG. 12, and the auxiliary information encoding module 1802 is used to execute the related content of step S1202 in the embodiment shown in FIG.
  • the patch information acquisition module 1801 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 1802 may correspond to the auxiliary information encoding module 109 in FIG. 2
  • the function of the patch information acquisition module 1801 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2
  • the function of the auxiliary information generating module 1802 may be implemented by the auxiliary information encoding module 109 in FIG.
  • the point cloud block information generation module 101 is used to obtain three-dimensional information, normal coordinate axis information, first identification and second identification of the patch to be encoded, and the packaging module 102 is used to obtain rotation transformation information of the patch to be encoded , Refer to patch identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • the decoding apparatus 1900 may include:
  • the auxiliary information decoding module 1901 is used to parse the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame;
  • the auxiliary information decoding module 1901 is further configured to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value;
  • the auxiliary information decoding module 1901 is further configured to determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is the fourth value; when the second identifier is the fifth value, from the code stream Parse the auxiliary information of the patch to be decoded, and the auxiliary information includes the three-dimensional information of the patch to be decoded;
  • the auxiliary information decoding module 1901 is also used for when the patch to be decoded is the n-th patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of the first-type patch in the reference point cloud frame of the current point cloud frame Parse the second identifier of the patch to be decoded from the code stream;
  • the auxiliary information decoding module 1901 is further configured to determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is the fourth value; when the second identifier is the fifth value, from the code stream Parse the auxiliary information of the patch to be decoded, and the auxiliary information includes the three-dimensional information of the patch to be decoded;
  • the reconstruction module 1902 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information decoding module 1901 is used to execute the related content of steps S1301-S1305 in the embodiment shown in FIG. 13, and the reconstruction module 1902 is used to execute the related content of step S1306 in the embodiment shown in FIG. 13.
  • the auxiliary information decoding module 1901 in FIG. 19 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1902 in FIG. 19 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 1901 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1902 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6.
  • this application is not limited to this.
  • FIG. 20 is a schematic structural diagram of the first point cloud encoding device provided by an embodiment of the application.
  • the encoding device 2000 may include:
  • the patch information acquisition module 2001 is used to acquire auxiliary information of the point cloud block patch to be encoded and the first identifier of the patch to be encoded;
  • the auxiliary information encoding module 2002 is configured to encode the first identifier of the patch to be encoded into the code stream when the first identifier is the first value;
  • the auxiliary information encoding module 2002 is further configured to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the second value or the third value;
  • the first identifier is the first value to indicate that the patch to be encoded is the first type of patch
  • the first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;
  • the first identifier is a third value to indicate that the patch to be encoded is a third type of patch.
  • the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.
  • the patch information acquisition module 2001 is used to execute the related content of step S1401 in the embodiment shown in FIG. 14, and the auxiliary information encoding module 2002 is used to execute the related content of step S1402 and step S1403 in the embodiment shown in FIG. 14.
  • the patch information acquisition module 2001 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 2002 may correspond to the auxiliary information encoding module 109 in FIG. 2
  • the function of the patch information acquisition module 2001 can be implemented by the point cloud block information generation module 101 and the packing module 102 in FIG. 2
  • the function of the auxiliary information generation module 2002 can be implemented by the auxiliary information encoding module 109 in FIG. 2.
  • the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded
  • the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc.
  • this application is not limited to this.
  • FIG. 21 is a schematic structural diagram of a first point cloud decoding apparatus provided by an embodiment of this application.
  • the decoding apparatus 2100 may include:
  • the auxiliary information decoding module 2101 is used to parse the first identifier of the point cloud block patch to be decoded from the code stream;
  • the auxiliary information decoding module 2101 is further configured to determine auxiliary information of the patch to be decoded according to auxiliary information of the reference patch of the patch to be decoded when the first identifier is the first value.
  • the auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information , Rotation transformation information and normal coordinate axis information;
  • the auxiliary information decoding module 2101 is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the second value.
  • the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and references to the patch to be decoded. Patch identification information;
  • the auxiliary information decoding module 2101 is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the third value.
  • the auxiliary information of the patch to be decoded includes 2D information, 3D information, rotation transformation information, and method. To coordinate axis information;
  • the reconstruction module 2102 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.
  • the auxiliary information decoding module 2101 is further configured to:
  • the reconstruction module 2102 is also used to:
  • the patch to be decoded is reconstructed.
  • the auxiliary information decoding module 2101 is used to execute the related content of steps S1501-S1504 in the embodiment shown in FIG. 15, and the reconstruction module 2102 is used to execute the related content of step S1506 in the embodiment shown in FIG. 15.
  • the auxiliary information decoding module 2101 in FIG. 21 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 2102 in FIG. 21 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5.
  • the function of the auxiliary information decoding module 2101 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 2102 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6.
  • this application is not limited to this.
  • FIG. 22 is a schematic block diagram of an implementation manner of a decoding device 2200 used in an embodiment of the present application.
  • the acquiring apparatus 2200 may include a processor 2201, a memory 2202, and a bus system 2203.
  • the processor 2201 and the memory 2202 are connected through a bus system 2203.
  • the memory 2202 is used to store instructions.
  • the processor 2201 is used to execute the instructions stored in the memory 2202 to execute various point cloud encoding or decoding methods described in this application. , Especially the method of filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.
  • the processor 2201 may be a central processing unit (CPU), and the processor 2201 may also be other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 2202 may include a ROM device or a RAM device. Any other suitable type of storage device can also be used as the memory 2202.
  • the memory 2202 may include code and data 22021 accessed by the processor 2201 using the bus 2203.
  • the memory 2202 may further include an operating system 22023 and an application program 22022.
  • the application program 22022 includes at least one of the point cloud encoding or decoding methods described in this application (especially the point cloud encoding and decoding methods described in this application) that allows the processor 2201 to execute program.
  • the application program 22022 may include applications 1 to N, which further include a point cloud encoding or decoding application (referred to as a point cloud decoding application) that executes the point cloud encoding or decoding method described in this application.
  • the bus system 2203 may also include a power bus, a control bus, and a status signal bus.
  • various buses are marked as the bus system 2203 in the figure.
  • the encoding apparatus 2200 may further include one or more output devices, such as a display 2204.
  • the display 2204 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input.
  • the display 2204 may be connected to the processor 2201 via a bus 2203.
  • the decoding device 2200 can execute the point cloud encoding method in this application, and can also execute the point cloud decoding method in this application.
  • the computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (for example, according to a communication protocol) .
  • computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves.
  • Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is properly termed a computer-readable medium.
  • any connection is properly termed a computer-readable medium.
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave to transmit instructions from a website, server, or other remote source
  • coaxial cable Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media.
  • magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
  • DSP digital signal processors
  • ASIC application-specific integrated circuits
  • FPGA field programmable logic arrays
  • processor may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein.
  • the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec.
  • the technology may be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.
  • the technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets).
  • ICs integrated circuits
  • a set of ICs for example, chipsets.
  • Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units.
  • various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

La présente invention se rapporte au domaine du traitement de données, et concerne un procédé d'encodage de nuage de points, un procédé de décodage de nuage de points, des appareils et un support de stockage. Le procédé consiste à, premièrement, obtenir des informations auxiliaires d'une parcelle à encoder et un premier identifiant de ladite parcelle. Comme le nombre de parcelles de premier type dans un groupe de nuage de points actuel peut être identique dans chaque trame de nuage de points, si le premier identifiant est une première valeur, et ladite parcelle est une première parcelle dans une trame de nuage de points actuelle, les informations auxiliaires et le premier identifiant de ladite parcelle peuvent être encodés dans un flux de code. Si le premier identifiant est la première valeur, et ladite parcelle n'est pas la première parcelle dans la trame de nuage de points actuelle, les informations auxiliaires de ladite parcelle peuvent être encodées dans le flux de code, et il n'est pas nécessaire d'encoder le premier identifiant de ladite parcelle dans le flux de code, ce qui simplifie le format d'encodage dans flux de code, réduit le temps système par bit du flux de code, et améliore l'efficacité d'encodage.
PCT/CN2020/083492 2019-04-30 2020-04-07 Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage WO2020220941A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910364032.2 2019-04-30
CN201910364032.2A CN111866484B (zh) 2019-04-30 2019-04-30 点云编码方法、点云解码方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2020220941A1 true WO2020220941A1 (fr) 2020-11-05

Family

ID=72965202

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/083492 WO2020220941A1 (fr) 2019-04-30 2020-04-07 Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage

Country Status (2)

Country Link
CN (1) CN111866484B (fr)
WO (1) WO2020220941A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7143115B2 (en) * 2003-04-15 2006-11-28 Pocket Soft, Inc. Method and apparatus for finding differences between two computer files efficiently in linear time and for using these differences to update computer files
CN104298971A (zh) * 2014-09-28 2015-01-21 北京理工大学 一种3d点云数据中的目标识别方法
US20180053040A1 (en) * 2016-08-19 2018-02-22 Umm Al-Qura University System and method for 3d local surface matching
WO2018102190A1 (fr) * 2016-11-29 2018-06-07 Blackmore Sensors and Analytics Inc. Procédé et système de classification d'un objet dans un ensemble de données en nuage de points
US20190087979A1 (en) * 2017-09-18 2019-03-21 Apple Inc. Point cloud compression

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1110957C (zh) * 1995-07-21 2003-06-04 大宇电子株式会社 用运动模型的分层结构分割与估算移动目标的运动的方法
US20130095920A1 (en) * 2011-10-13 2013-04-18 Microsoft Corporation Generating free viewpoint video using stereo imaging
CN103916665B (zh) * 2013-01-07 2018-05-29 华为技术有限公司 一种图像的解码、编码方法及装置
CN103218817B (zh) * 2013-04-19 2015-11-18 深圳先进技术研究院 植物器官点云的分割方法和系统
CN103942757B (zh) * 2014-03-19 2017-08-29 王斌君 基于内容特征的图像碎片匹配方法和系统
CN105491414B (zh) * 2015-11-19 2017-05-17 深圳市鹰硕技术有限公司 一种图像同步显示方法及装置
CN105825207B (zh) * 2016-04-20 2019-05-14 北京航空航天大学 片段化的高压线探测方法和装置
CN107230225B (zh) * 2017-04-25 2020-06-09 华为技术有限公司 三维重建的方法和装置
EP3429206A1 (fr) * 2017-07-13 2019-01-16 Thomson Licensing Procédé et dispositif de codage d'un nuage de points
CN108680100B (zh) * 2018-03-07 2020-04-17 福建农林大学 三维激光点云数据与无人机点云数据匹配方法
CN108632607B (zh) * 2018-05-09 2019-06-21 北京大学深圳研究生院 一种基于多角度自适应帧内预测的点云属性压缩方法
CN111479114B (zh) * 2019-01-23 2022-07-22 华为技术有限公司 点云的编解码方法及装置
CN118042135A (zh) * 2019-03-19 2024-05-14 华为技术有限公司 点云编码方法、点云解码方法、装置及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7143115B2 (en) * 2003-04-15 2006-11-28 Pocket Soft, Inc. Method and apparatus for finding differences between two computer files efficiently in linear time and for using these differences to update computer files
CN104298971A (zh) * 2014-09-28 2015-01-21 北京理工大学 一种3d点云数据中的目标识别方法
US20180053040A1 (en) * 2016-08-19 2018-02-22 Umm Al-Qura University System and method for 3d local surface matching
WO2018102190A1 (fr) * 2016-11-29 2018-06-07 Blackmore Sensors and Analytics Inc. Procédé et système de classification d'un objet dans un ensemble de données en nuage de points
US20190087979A1 (en) * 2017-09-18 2019-03-21 Apple Inc. Point cloud compression

Also Published As

Publication number Publication date
CN111866484B (zh) 2023-06-20
CN111866484A (zh) 2020-10-30

Similar Documents

Publication Publication Date Title
JP7057453B2 (ja) 点群符号化方法、点群復号化方法、符号器、及び復号器
WO2020151496A1 (fr) Procédé et appareil de codage/décodage de nuage de points
CN110971912B (zh) 点云编解码方法、编解码器、编解码装置和存储介质
US11961265B2 (en) Point cloud encoding and decoding method and apparatus
EP4138397A1 (fr) Appareil de transmission de données de nuage de points, procédé de transmission de données de nuage de points, appareil de réception de données de nuage de points et procédé de réception de données de nuage de points
CN110944187B (zh) 点云编码方法和编码器
WO2020147379A1 (fr) Procédé et dispositif de filtrage de nuage de points et support de stockage
CN117978993A (zh) 对点云数据进行编码/解码的设备
CN117978996A (zh) 点云数据发送设备和方法、点云数据接收设备和方法
US20220007037A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
JP5897236B1 (ja) 復号器及び符号器、並びに映像シーケンスを符号化する方法
JP7271671B2 (ja) 点群符号化方法及びエンコーダ
WO2020220941A1 (fr) Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage
WO2020187283A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage
US20230156229A1 (en) Methods and apparatus to signal enabled views per atlas in immersive video
CN113453006B (zh) 一种图片封装方法、设备以及存储介质
WO2020015517A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points et décodeur
US20240129537A1 (en) Method and apparatus for signaling cmaf switching sets in isobmff
US20230239508A1 (en) Methods and apparatus to identify a video decoding error

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20799242

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20799242

Country of ref document: EP

Kind code of ref document: A1