WO2020151496A1 - Procédé et appareil de codage/décodage de nuage de points - Google Patents

Procédé et appareil de codage/décodage de nuage de points Download PDF

Info

Publication number
WO2020151496A1
WO2020151496A1 PCT/CN2020/071022 CN2020071022W WO2020151496A1 WO 2020151496 A1 WO2020151496 A1 WO 2020151496A1 CN 2020071022 W CN2020071022 W CN 2020071022W WO 2020151496 A1 WO2020151496 A1 WO 2020151496A1
Authority
WO
WIPO (PCT)
Prior art keywords
patch
information
matching
decoded
index
Prior art date
Application number
PCT/CN2020/071022
Other languages
English (en)
Chinese (zh)
Inventor
张德军
蔡康颖
刘波
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020151496A1 publication Critical patent/WO2020151496A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the embodiments of the present application provide a point cloud encoding and decoding method and device, which are beneficial to make the generated depth map and texture map smaller in size, reduce the bit overhead of the code stream, and thereby improve the encoding and decoding efficiency.
  • an embodiment of the present application provides a point cloud encoding method, including:
  • the side information of the patch to be coded if the patch to be coded is the patch in the first frame of the point cloud in the subgroup, then the side information of the patch to be coded is encoded into the code stream; the side information includes 2D information, 3D information, and rotation transformation Information, global matching patch identification and matching patch index; the global matching patch identification is used to indicate whether the patch to be encoded is a global matching patch in the subgroup; if the patch to be encoded is a patch in the point cloud of the non-first frame in the subgroup, the The side information of the patch to be encoded is compiled into the code stream, and the side information includes the three-dimensional information and the matching patch index.
  • the global matching patch identifier of the patch (for example, patch A) in the first frame of point cloud is encoded into the code stream in order to parse out the global matching patch identifier of patch A at the decoding end.
  • patch A is used as For reference patches of other patches (such as patch B in the point cloud of the first frame)
  • the global matching patch identifier of patch B can be determined, and then if the global matching patch identifier of patch B is the first value ( For example, 1 or true), the two-dimensional information, normal coordinate axis information, and rotation transformation information of patch B's reference patch (that is, patch A) can be derived or determined respectively to derive or determine patch B's two-dimensional information, normal coordinate axis information, and Rotate and transform the information, so as to avoid encoding the aforementioned information in the code stream during encoding, thereby reducing the bit overhead of the code stream and improving the coding efficiency at the same time.
  • the first frame point cloud of the subgroup refers to the 0th frame point cloud in the subgroup
  • the global matching patch identifier of the patch to be encoded can be a first value (such as true or 1) or a second value (such as false or 0).
  • the first value is used to indicate that the patch to be coded is a global matching patch in the subgroup
  • the second value is used to indicate that the patch to be coded is not a global matching patch in the subgroup.
  • the patch to be coded is a global matching patch in the subgroup refers to the point cloud (other point cloud frames) other than the point cloud (also known as a point cloud or point cloud frame) to which the patch to be coded belongs in the subgroup
  • a patch that matches the patch to be coded can be found in all of them (hereinafter may be referred to as a reference patch).
  • the patch to be coded is not a global matching patch in the subgroup means that the patch to be coded can find the patch to be coded in the part of the point cloud (that is, some point cloud frames) other than the point cloud to which the patch to be coded belongs in the subgroup.
  • the matching patch index of the patch to be coded is the index of the patch (which may be referred to as the reference patch hereinafter) that matches the patch to be coded in the reference frame of the point cloud to which the patch to be coded belongs.
  • the matching patch index of the patch to be encoded can be a non-default value or a default value.
  • the index of the patch to be matched is a non-default value to indicate the reference patch that matches the patch to be encoded.
  • the patch to be encoded is a matching patch. For example, patch[k].matchedPatchIndex is not equal to -1, and -1 is the default value, which represents the reference patch of the patch to be encoded.
  • the index of the patch to be matched is the default value to indicate that the patch to be coded does not have a matching patch.
  • the patch to be coded is a non-matching patch.
  • patch[k].matchedPatchIndex is equal to -1, and -1 is the default value, which means that the patch to be encoded does not have a matching patch.
  • the patch in the reference frame indicated by the matching patch index of the patch to be encoded is the reference patch.
  • the normal coordinate axis information may also be referred to as normal axis information, such as normalAxis, which is the normal axis index of the patch to be encoded.
  • the rotation transformation information of the patch to be encoded may be rotation angle information, coordinate axis transformation information or mirror transformation information.
  • the rotation angle information of the patch to be encoded includes a specific rotation angle value or a rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch is rotated by 20°; if the rotation angle value of the patch to be coded is 80, It means that the patch is rotated by 80°.
  • the rotation angle index of the patch to be coded is 0, it means that the patch does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch is rotated by 15°; if the rotation angle index of the patch to be coded is 2, It means that the patch is rotated by 30°; if the rotation angle index of the patch to be encoded is 3, it means that the patch is rotated by 45°.
  • rotation of the patch to be encoded may be clockwise or counterclockwise, which is not limited in this application.
  • the coordinate axis transformation information of the patch to be encoded can be expressed as the coordinate axis transformation index.
  • the coordinate axis exchange flag of the patch to be encoded is 0 or false, it means that the tangent axis and the double tangent axis of the patch to be encoded are not exchanged;
  • the coordinate axis exchange flag of the coded patch is 1 or true, it means that the tangent coordinate axis and the double tangent coordinate axis of the patch to be coded are exchanged. For example, suppose the U-axis index is 0 and the V-axis index is 1.
  • the index of the tangent axis of the patch to be encoded is 0, indicating that the tangent axis of the patch to be encoded is the U axis; the double tangent of the patch to be encoded The index of the coordinate axis is 1, indicating that the double tangent coordinate axis of the patch to be encoded is the V axis.
  • the coordinate axis exchange flag of the patch to be coded is false, the index of the tangent coordinate axis of the patch and the index of the double tangent coordinate axis are not exchanged; if the coordinate axis exchange flag of the patch to be coded is true, the tangent coordinate of the patch to be coded is exchanged
  • the index of the axis and the index of the double tangent axis that is, the index of the tangent axis of the patch to be coded is updated to 1, and the index of the double tangent axis of the patch is updated to 0.
  • the mirror transformation information can be expressed as a mirror transformation index.
  • the mirror transformation index when the mirror transformation index is 0, it means that the U axis is transformed into a -U axis; for example, when the mirror transformation index is 1, it means that the V axis is transformed into a -V axis; such as the mirror transformation index.
  • it When it is 2, it means that the V axis is converted to -V axis, and the U axis is converted to -U axis.
  • the three-dimensional information of the patch to be coded includes u1, v1, and d1, or the three-dimensional information of the patch to be coded includes the difference between u1 of the patch to be coded and u1 of the reference patch, v1 of the patch to be coded and v1 of the reference patch The difference between and the difference between the d1 of the patch to be encoded and the d1 of the reference patch.
  • u1 also called patch_3d_shift_tangent_axis
  • v1 also called patch_3d_shift_bitangent_axis
  • d1 also called patch_3d_shift_normal_axis
  • the two-dimensional information of the patch to be encoded includes u0, v0, sizeU0, and sizeV0, or the two-dimensional information of the patch to be encoded includes the difference between u0 of the patch to be encoded and u0 of the reference patch, and the v0 of the patch to be encoded and The difference between v0 of the reference patch, the difference between the sizeU0 of the patch to be encoded and the sizeU0 of the reference patch, and the difference between the sizeV0 of the patch to be encoded and the sizeV0 of the reference patch.
  • u0 (also known as patch_2d_shift_u) is the abscissa value x of the position of the occupancy map of the patch to be encoded in the point cloud occupancy map to which it belongs
  • v0 (also called patch_2d_shift_v) is the occupancy map of the patch to be encoded at the point where it belongs
  • the ordinate value y of the position in the cloud occupancy map sizeU0 is the width of the patch to be encoded (for example, the width of the patch on the point cloud occupancy map to which it belongs); sizeV0 is the height of the patch to be encoded (for example, the patch is in the point cloud to which it belongs) Take up the height on the map).
  • the patch to be coded is a patch in the first frame of the point cloud in the subgroup, and the patch to be coded is a global matching patch and the patch to be coded has a patch that matches it, the edges of the patch to be coded Information is compiled into the code stream, and the matching patch index of the patch to be encoded is a non-default value to indicate the reference patch that matches the patch to be encoded; the global matching patch identifier of the patch to be encoded is the first value to indicate the patch to be encoded Patch is the global matching patch in the subgroup;
  • the global matching patches in the subgroup have the same two-dimensional information, normal coordinate axis information, rotation transformation information, and global matching patch identification, for the patch in the first frame point cloud in the subgroup (for example, patch A and patch A
  • the two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identifier and matching patch index of patch A are compiled into the code stream.
  • the patch for example, patch B
  • the two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index of patch A are parsed from the code stream, and the three-dimensional information and matching patch index of patch B are parsed from the code stream.
  • the matching patch index of patch B it is determined that patch A is the reference patch of patch B, and then the global matching patch identifier of patch B is determined according to the global matching patch identifier of patch A. If the global matching patch identifier of patch B is the first value (for example, l or true), the two-dimensional information, normal coordinate axis information, and rotation transformation information of patch B are determined according to the reference patch of patch B (that is, patch A).
  • the side information of the patch to be coded is encoded into the code stream, including:
  • the side information of the patch to be coded also includes normal coordinate axis information
  • the side information of the patch to be encoded is compiled into the code stream
  • the matching patch index is a default value to indicate that the patch to be encoded does not have a matching patch.
  • patch[k].matchedPatchIndex is equal to -1
  • -1 is the default value, which means that the patch to be encoded has no matching patch.
  • the global matching patch identifier of the patch to be encoded is the first value to indicate that the patch to be encoded is a global matching patch in the subgroup.
  • the side information of the patch to be coded is encoded into the code stream, including: if the patch to be coded is the first frame point in the subgroup The patch in the cloud, and the patch to be coded is a non-global matching patch and the patch to be coded has a matching reference patch, and the side information of the patch to be coded is compiled into the code stream; the matching patch index of the patch to be coded is non-default Value to indicate the reference patch that matches the patch to be encoded.
  • patch[k].matchedPatchIndex is not equal to -1 (for example, -1 is the default value), indicating the reference patch of the patch to be encoded.
  • the global matching patch identifier of the patch to be coded is a second value to indicate that the patch to be coded is not a global matching patch in the subgroup.
  • the patches that match each other in the subgroup have the same normal coordinate axis information and rotation transformation information
  • patch in the first frame point cloud in the subgroup for example, patch A and patch A are non-global matching patches and patch A has The matching patch B.
  • the two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identifier and matching patch index of patch A are compiled into the code stream.
  • patch B the two-dimensional information, three-dimensional information and matching patch index of patch B are compiled into the code stream.
  • the normal coordinates of patch B are determined according to the normal coordinate axis information and rotation transformation information of the reference patch of patch B (that is, patch A) Axis information and rotation transformation information, so as to avoid coding the aforementioned information in the code stream, thereby also reducing the bit overhead of the code stream and improving the coding efficiency at the same time.
  • the normal coordinate axis information of patch A does not need to be encoded into the code stream, but on the decoding side, patch A is determined according to the normal coordinate axis information of the reference patch indicated by the matching patch index of patch A.
  • the normal axis information does not need to be encoded into the code stream, but on the decoding side, patch A is determined according to the normal coordinate axis information of the reference patch indicated by the matching patch index of patch A. The normal axis information.
  • the side information of the patch to be coded also includes normal coordinate axis information (for example, normalAxis, that is, the patch to be coded) Normal axis index), encode the side information of the patch to be encoded into the code stream;
  • the matching patch index of the patch to be encoded is the default value to indicate that the patch to be encoded does not have a matching patch.
  • patch[k].matchedPatchIndex is equal to -1, and -1 is the default value, which means that the patch to be encoded has no matching patch.
  • the global matching patch identifier of the patch to be encoded is the second value to indicate that the patch to be encoded is not a global matching patch in the subgroup.
  • the side information of the patch to be coded also includes two-dimensional information and normal axis information (for example, normalAxis, which is the normal axis of the patch to be coded) Index) and rotation transformation information, the side information of the patch to be encoded is encoded into the code stream, where the matching patch index of the patch to be encoded is a default value to indicate that the patch to be encoded does not have a matching patch.
  • patch[k].matchedPatchIndex is equal to -1, and -1 is the default value, which means that the patch to be encoded has no matching patch.
  • the purpose of encoding the global matching patch identifier of the patch into the code stream is that patches that have a matching relationship with each other have the same global matching patch identifier. If patch A is a reference patch for other patches (such as patch B in the first frame), even if the global matching patch identifier of patch B is not parsed from the code stream at the decoder, it can be determined based on the global matching patch identifier of patch A Patch B's global matching patch identifier.
  • the two-dimensional information and/or three-dimensional information of the patch to be encoded is encoded into the code stream by differential encoding.
  • differential encoding to encode two-dimensional information into the code stream is specifically the difference between u0 of the patch to be encoded and u0 of the reference patch, and between v0 of the patch to be encoded and v0 of the reference patch
  • the difference between the sizeU0 of the patch to be encoded and the sizeU0 of the reference patch and the difference between the sizeV0 of the patch to be encoded and the sizeV0 of the reference patch are encoded into the code stream; similarly, differential encoding is adopted
  • Encoding three-dimensional information into the code stream is specifically the difference between u0 of the patch to be encoded and u0 of the reference patch, the difference between v0 of the patch to be encoded and v0 of the reference patch, and the sizeU0 of the patch to be encoded
  • the difference between the sizeU0 of the reference patch and the difference between the sizeV0 of the patch to be encoded and the sizeV0 of the reference patch is encoded into the code stream.
  • the use of differential encoding can further reduce the bit overhead of
  • the reference patch of the patch to be coded is the patch indicated by the matching patch index of the patch to be coded in the point cloud to which the patch to be coded belongs.
  • the embodiment of the present application discards the matching relationship between the subgroups that is used to constrain the rotation transformation information of the matching patch, so that the packing of each subgroup is more flexible, and the generated depth map and texture map are smaller in size. Because part of the side information of the global matching patch is encoded only once, the purpose of improving the encoding efficiency is achieved and the bit overhead of the code stream is reduced at the same time.
  • an embodiment of the present application provides a point cloud decoding method, including:
  • the side information of the patch to be decoded Analyze the side information of the patch to be decoded from the code stream; if the side information includes two-dimensional information, three-dimensional information, rotation transformation information, global matching patch identification and matching patch index, the normal direction of the patch to be decoded is obtained according to the matching patch index Coordinate axis information; if the side information includes three-dimensional information and matching patch index, the global matching patch identifier, two-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded are obtained according to the matching patch index; according to the second of the patch to be decoded
  • the dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information reconstruct the patch to be decoded; wherein the global matching patch identifier is used to indicate whether the patch to be decoded is a global matching patch in the subgroup.
  • the normal coordinate axis information of A can also be derived from the normal coordinate axis information of the reference patch indicated by the matching patch index of patch A on the decoding side.
  • the normal coordinate axis information of the patch to be decoded is obtained according to the matching patch index, including :
  • the matching patch index is a default value to indicate that the patch to be decoded does not have a matching patch
  • the global matching patch identifier is the first value to indicate that the patch to be decoded is a global matching patch in the subgroup.
  • the matching patch index of the patch to be decoded is a non-default value to indicate the reference patch of the patch to be decoded;
  • the global matching patch identifier of the patch to be decoded is the second value to indicate that the patch to be decoded is not a global matching patch in the subgroup .
  • Decode v0 of the patch obtain the sizeU0 of the patch to be decoded according to the difference between the sizeU0 of the patch to be decoded and the sizeU0 of the reference patch and the sizeU0 of the reference patch, and obtain the sizeU0 of the patch to be decoded according to the sum of the difference between the sizeV0 of the patch to be decoded and the sizeV0 of the reference patch Refer to the sizeV0 of the patch to obtain the sizeV0 of the patch to be decoded.
  • an embodiment of the present application provides another point cloud decoding method, including:
  • the auxiliary information encoding module is specifically used to:
  • the patch in the point cloud of the first frame, and the patch to be coded is a global matching patch and the patch to be coded has a matching patch, and the side information of the patch to be coded is compiled into the code stream;
  • the matching patch index of the patch to be coded is non The default value to indicate the reference patch that matches the patch to be encoded.
  • the auxiliary information encoding module is specifically used to:
  • the patch in the first frame of the point cloud, and the patch to be encoded is a non-global matching patch and the patch to be encoded has a matching patch.
  • the side information of the patch to be encoded also includes two-dimensional information, and the side information of the patch to be encoded is included Code stream; where the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch that matches the patch to be encoded.
  • the auxiliary information encoding module is specifically used for:
  • the three-dimensional information includes u1, d1 and v1.
  • u1 also called patch_3d_shift_tangent_axis
  • v1 also called patch_3d_shift_bitangent_axis
  • D1 also called patch_3d_shift_normal_axis
  • an embodiment of the present application also provides a point cloud decoding device, including:
  • the auxiliary information decoding module is used to parse the side information of the patch to be decoded from the code stream;
  • the reconstruction module is used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded; wherein the global matching patch identifier is used to indicate whether the patch to be decoded is in a subgroup The global matching patch.
  • the auxiliary information decoding module is specifically used for:
  • the auxiliary information decoding module Specifically used: if it is determined that the matching patch index of the patch to be decoded is a non-default value, the global matching patch identifier of the reference patch indicated by the matching patch index is used to determine the global matching patch identifier of the patch to be decoded; if the global matching patch identifier of the patch to be decoded If the matching patch ID is the first value, the two-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded are determined according to the two-dimensional information, normal coordinate axis information, and rotation transformation information of the reference patch; The matching patch index of the patch is the default value to indicate the reference patch of the patch to be decoded, and the global matching patch identifier of the patch to be decoded
  • the global matching patch identifier of the reference patch indicated by the matching patch index is determined to determine the global matching patch identifier of the patch to be decoded; if the global matching patch identifier of the patch to be decoded is determined Is the second value; then the two-dimensional information of the patch to be decoded is also parsed from the code stream; and the normal coordinate axis of the patch to be decoded is determined according to the normal coordinate axis information and the rotation transformation information of the reference patch indicated by the matching patch index Information and rotation transformation information; where the matching patch index of the patch to be decoded is a non-default value to indicate the reference patch of the patch to be decoded, and the global matching patch identifier of the patch to be decoded is the second value to indicate that the patch to be decoded is not a child
  • the reference patch is the patch indicated by the matching patch index in the reference frame of the point cloud to which the
  • the auxiliary information decoding module is specifically used for:
  • the normal coordinate axis information of the patch to be decoded is also parsed from the code stream;
  • the auxiliary information decoding module Specifically used: if it is determined that the matching patch index of the patch to be decoded is the default value, then the two-dimensional information, normal coordinate axis information, and rotation transformation information are also analyzed from the code stream, and the global matching patch identifier of the patch to be decoded is determined as Second value
  • the matching patch index of the patch to be decoded is the default value to indicate that the patch to be decoded has no matching patch
  • the global matching patch identifier of the patch to be decoded is the second value to indicate that the patch to be decoded is not in the subgroup Match patch globally.
  • the two-dimensional information and/or three-dimensional information of the patch to be decoded is encoded into the code stream by differential decoding; wherein the two-dimensional information includes u0, v0, sizeU0, sizeV0, and u0 is the patch to be decoded.
  • an embodiment of the present application provides another point cloud encoding device, including:
  • the patch information acquisition module is used to acquire the side information of the patch to be encoded
  • the auxiliary information encoding module is used to encode the instruction information into the code stream, the instruction information is used to indicate whether to encode the side information of the patch to be encoded according to the target encoding method; the target encoding method includes the point cloud according to the first aspect of the claim Part or all of the encoding method.
  • an embodiment of the present application provides another point cloud decoding device, including:
  • a device for encoding point cloud data may include a memory and an encoder.
  • the memory is used to store point cloud data.
  • the encoder is used to implement the point cloud encoding method provided in the first aspect or any possible design of the first aspect.
  • a device for decoding point cloud data may include a memory and a decoder.
  • the memory is used to store point cloud data in the form of code stream.
  • the decoder is used to implement the point cloud encoding method provided by the second aspect or any possible design of the second aspect.
  • an encoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the point cloud encoding method provided in the first aspect.
  • the present application also provides a computer-readable storage medium, including program code, when the program code runs on a computer, the computer executes the point cloud decoding method provided in the fourth aspect.
  • FIG. 2 is a schematic block diagram of an encoder that can be used in an example of an embodiment of the present application
  • FIG. 3 is a schematic diagram of a point cloud, a patch of the point cloud, and an occupancy map of the point cloud that can be used in an embodiment of the present application;
  • FIG. 4 is a schematic block diagram of a decoder that can be used in an example of an embodiment of the present application
  • FIG. 5 is a schematic diagram of subgroups in an embodiment of the application.
  • FIG. 6 is a schematic flowchart of a point cloud encoding method provided by an embodiment of the application.
  • FIG. 8 is a schematic diagram of a specific point cloud coding and decoding provided by an embodiment of the application.
  • FIG. 9 is a schematic structural diagram of a point cloud encoding device provided by this application.
  • FIG. 10 is a schematic structural diagram of a point cloud decoding device provided by this application.
  • FIG. 11 is a schematic structural diagram of a point cloud encoding device provided by this application.
  • FIG. 12 is a schematic structural diagram of a point cloud decoding device provided by this application.
  • FIG. 13 is a schematic block diagram of an implementation manner of a decoding device according to an embodiment of the application.
  • At least one (species) in the embodiments of the present application includes one (species) or more (species).
  • Multiple (species) refers to two (species) or more than two (species).
  • at least one of A, B, and C includes: A alone, B alone, A and B at the same time, A and C at the same time, B and C at the same time, and A, B, and C at the same time.
  • the term "and/or" in the embodiments of the present application is merely an association relationship describing associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean that there is A alone, and both A and B, there are three cases of B alone.
  • character "/" in the embodiments of the present application generally indicates that the associated objects before and after are in an "or" relationship.
  • the character “/” means division operation, such as A/B means A divided by B.
  • the terms “first” and “second” in the embodiments of the present application are used to distinguish different objects, and do not limit the order of the different objects.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system 1 that can be used in an example of an embodiment of the present application.
  • the term "point cloud coding" or “coding” may generally refer to point cloud encoding or point cloud decoding.
  • the encoder 100 of the point cloud decoding system 1 can encode the point cloud to be coded according to any point cloud coding method proposed in this application.
  • the decoder 200 of the point cloud decoding system 1 can decode the point cloud to be decoded according to the point cloud decoding method corresponding to the point cloud encoding method used by the encoder proposed in this application.
  • the source device 10 and the destination device 20 may include various devices, including desktop computers, mobile computing devices, notebook (for example, laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart" phones.
  • desktop computers mobile computing devices
  • notebook (for example, laptop) computers tablet computers
  • set-top boxes telephone handsets such as so-called “smart” phones.
  • Computer television, camera, display device, digital media player, video game console, onboard computer or the like.
  • Destination device 20 may receive encoded point cloud data from source device 10 via link 30.
  • the link 30 may include one or more media or devices capable of moving the encoded point cloud data from the source device 10 to the destination device 20.
  • the link 30 may include one or more communication media that enable the source device 10 to send the encoded point cloud data directly to the destination device 20 in real time.
  • the source device 10 may modulate the coded point cloud data according to a communication standard (for example, a wireless communication protocol), and may transmit the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and/or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet).
  • the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from source device 10 to destination device 20.
  • the encoded data may be output from the output interface 140 to the storage device 40.
  • the encoded point cloud data can be accessed from the storage device 40 through the input interface 240.
  • the storage device 40 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital versatile discs (DVD), and compact disc read-only discs. only memory, CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded point cloud data.
  • the storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded point cloud data generated by the source device 10.
  • the destination device 20 can access the stored point cloud data from the storage device 40 via streaming or downloading.
  • the file server may be any type of server capable of storing the encoded point cloud data and transmitting the encoded point cloud data to the destination device 20.
  • Example file servers include network servers (for example, for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives.
  • the destination device 20 can access the coded point cloud data through any standard data connection (including an Internet connection).
  • the transmission of coded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.
  • the point cloud decoding system 1 illustrated in FIG. 1 is only an example, and the technology of this application can be applied to point cloud decoding that does not necessarily include any data communication between a point cloud encoding device and a point cloud decoding device (for example, point cloud decoding).
  • Cloud coding or point cloud decoding) device In other instances, the data is retrieved from local storage, streamed on the network, and so on.
  • the point cloud encoding device may encode data and store the data to the memory, and/or the point cloud decoding device may retrieve the data from the memory and decode the data. In many instances, encoding and decoding are performed by devices that do not communicate with each other but only encode data to and/or retrieve data from memory and decode the data.
  • the source device 10 includes a data source 110, an encoder 100 and an output interface 140.
  • the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter (or referred to as a transmitter).
  • the data source 110 may include a point cloud capture device (for example, a camera), a point cloud archive containing previously captured point cloud data, a point cloud feed interface to receive point cloud data from a point cloud content provider, and/or use A computer graphics system that generates point cloud data, or a combination of these sources of point cloud data.
  • the destination device 20 includes an input interface 240, a decoder 200 and a display device 120.
  • the input interface 240 includes a receiver and/or a modem.
  • the input interface 240 may receive coded point cloud data via the link 30 and/or from the storage device 40.
  • the display device 120 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 120 displays the decoded point cloud data.
  • the display device 120 may include various display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • the encoder 100 and the decoder 200 may be integrated with the audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams.
  • MUX-DEMUX multiplexer-demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
  • Each of the encoder 100 and the decoder 200 may be implemented as any one of a variety of circuits such as the following: one or more microprocessors, digital signal processing (DSP), application specific integrated circuits (application Specific integrated circuit, ASIC), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof.
  • DSP digital signal processing
  • ASIC application Specific integrated circuit
  • FPGA field-programmable gate array
  • the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors.
  • Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, any of which may be integrated as a combined encoder/decoder in the corresponding device Part of the device (codec).
  • This application may generally refer to the encoder 100 as “signaling” or “sending” certain information to another device such as the decoder 200.
  • the term “signaling” or “sending” may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. retrieve the syntax element at any time.
  • FIG. 2 it is a schematic block diagram of an encoder 100 that can be used as an example of the embodiment of the present application.
  • Fig. 2 illustrates an example of an MPEG (moving picture expert group) point cloud compression (PCC) coding framework.
  • the encoder 100 may include a patch information generating module 101, a packing module 102, a depth map generating module 103, a texture map generating module 104, a first filling module 105, an image or video-based encoding module 106, an occupancy Picture encoding module 107, auxiliary information encoding module 108, multiplexing module 109 and so on.
  • the encoder 100 may further include a point cloud filtering module 110, a second filling module 111, a point cloud reconstruction module 112, and so on. among them:
  • the patch information generating module 101 is used to divide a frame of point cloud to generate multiple patches by using a certain method, and to obtain related information of the generated patches.
  • patch refers to a collection of points in a frame of point cloud, and usually a connected area corresponds to a patch.
  • Patch related information may include but is not limited to at least one of the following information: the number of patches divided into the point cloud, the position information of each patch in the three-dimensional space, the index of the normal axis of each patch, The depth map generated by the projection of a patch from the three-dimensional space to the two-dimensional space, the depth map size of each patch (such as the width and height of the depth map), the occupancy map generated by the projection of each patch from the three-dimensional space to the two-dimensional space, etc.
  • the related information includes the number of patches that the point cloud is divided into, the index of the normal axis of each patch, the size of the depth map of each patch, and the number of patches in the point cloud.
  • the location information of each patch, the size information of the occupancy map of each patch, etc. can be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (ie, compression encoding).
  • the part of the related information such as part of the two-dimensional information (such as sizeU0 and sizeV0), three-dimensional information (such as u1, d1, and v1) of the patch generated by segmentation of the point cloud, or even the first part of the subgroup
  • the global matching patch identifier for example, patch[k].globalPatchFlag
  • the auxiliary information encoding module 108 can be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (ie, compression encoding).
  • part of the related information such as the three-dimensional information of the patch generated by segmentation of the point cloud (for example, u1, d1, and v1) can be sent as auxiliary information to the auxiliary information encoding module 108 for encoding ( That is compression coding).
  • the auxiliary information encoding module 108 can use different encoding methods (for example, fixed-length encoding or differential encoding) to encode or write side information into the code stream, which will be described in detail below in this application.
  • the occupancy map of each patch can be sent to the packaging module 102 for packaging.
  • the patches of the point cloud are arranged in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch; Then, according to the sequence of the arranged patches, the occupancy map of the patch is inserted into the available area of the point cloud occupancy map in turn to obtain the occupancy map of the point cloud.
  • the specific location information of each patch in the point cloud occupancy map and the depth map of each patch may be sent to the depth map generating module 103.
  • the packaging module 102 may respectively package the point clouds included in the multiple subgroups to obtain an occupancy map of each point cloud in the to-be-coded point cloud group.
  • the packing algorithm used when packing the point clouds in a subgroup is usually the same.
  • the packing algorithm used when packing the point clouds in different subgroups may be the same or different, which is not limited in the embodiment of the present application.
  • the occupancy map of the point cloud can be filled by the second filling module 111 and then sent to the occupancy map encoding module 107 for encoding.
  • the occupancy map of the point cloud may be used to instruct the depth map generating module 103 to generate the depth map of the point cloud and the texture map generating module 104 to instruct to generate the texture map of the point cloud.
  • the packing module 102 can also obtain the matching patch index (for example, patch[k].matchedPatchIndex), rotation transformation information (for example, patch[k].orientationIndex) and another part of two-dimensional information (for example, u0 and v0) of the patch.
  • the matching patch index of the patch for example, patch[k].matchedPatchIndex
  • rotation transformation information for example, patch[k].orientationIndex
  • another part of two-dimensional information for example, u0 and v0
  • FIG. 3 it is a schematic diagram of a point cloud, a patch of the point cloud, and an occupancy map of the point cloud applicable to the embodiments of the present application.
  • (a) in Figure 3 is a schematic diagram of a frame of point cloud
  • (b) in Figure 3 is a schematic diagram of the patch of the point cloud obtained based on (a) in Figure 3
  • (c) in Figure 3 is A schematic diagram of the occupancy map of the point cloud obtained by packaging the occupancy map of each patch obtained by mapping each patch shown in (b) on a two-dimensional plane.
  • the depth map generating module 103 is configured to generate a depth map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud, and depth information, and send the generated depth map to the first filling module 105 , To fill the blank pixels in the depth map to obtain a filled depth map.
  • the texture map generating module 104 is configured to generate a texture map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud, and texture information, and send the generated texture map to the first filling module 105 , To fill the blank pixels in the texture map to obtain the filled texture map.
  • the filled depth map and the filled texture map are sent by the first filling module 105 to the image or video-based encoding module 106 for image or video-based encoding.
  • the encoding result (ie, code stream) obtained by the image or video-based encoding module 106 is sent to the point cloud reconstruction module 112 for point cloud reconstruction to obtain a reconstructed point cloud (ie, to obtain a reconstructed point cloud).
  • Point cloud geometry information Specifically, video decoding is performed on the encoded depth map obtained by the image or video-based encoding module 106 to obtain the decoded depth map of the point cloud; the decoded depth map, the occupancy map of the point cloud and the auxiliary information of each patch are used , Obtain the reconstructed point cloud geometric information (for example, reconstruct the patch in the point cloud).
  • the geometric information of the point cloud refers to the coordinate values of points in the point cloud (for example, each point in the point cloud) in the three-dimensional space.
  • the point cloud reconstruction module 112 may also send the texture information of the point cloud and the reconstructed point cloud geometric information to the coloring module, and the coloring module is used to color the reconstructed point cloud to obtain the reconstructed point cloud. Texture information.
  • the texture map generating module 104 may also generate a texture map of the point cloud based on information obtained by filtering the reconstructed geometric information of the point cloud by the point cloud filtering module 110.
  • the encoder 100 shown in FIG. 2 is only an example. In a specific implementation, the encoder 100 may include more or less modules than those shown in FIG. 2. The embodiment of the present application does not limit this.
  • FIG. 4 it is a schematic block diagram of a decoder 200 that can be used as an example of an embodiment of the present application.
  • FIG. 4 uses the MPEG PCC decoding framework as an example for description.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud geometric information reconstruction module 205, and a point cloud
  • the cloud filtering module 206 and the point cloud texture information reconstruction module 207 among them:
  • the demultiplexing module 201 is used to send the input code stream (ie, the combined code stream) to the corresponding decoding module. Specifically, the code stream containing the coded texture map and the coded depth map is sent to the image or video-based decoding module 202; the code stream containing the coded occupancy map is sent to the occupancy map decoding module 203 , Sending the code stream containing the encoded auxiliary information to the auxiliary information decoding module 204.
  • the image or video-based decoding module 202 is used to decode the received encoded texture map and the encoded depth map; and send the decoded texture map information to the point cloud texture information reconstruction module 207, The decoded depth map information is sent to the geometric information reconstruction module 205 of the point cloud.
  • the occupancy map decoding module 203 is configured to decode the received bitstream containing the encoded occupancy map, and send the decoded occupancy map information to the geometric information reconstruction module 205 of the point cloud.
  • the auxiliary information decoding module 204 is configured to decode the received encoded auxiliary information, and send the decoded information indicating the auxiliary information to the geometric information reconstruction module 205 of the point cloud.
  • the auxiliary information decoding module 204 decodes to obtain side information, such as the two-dimensional information of the patch to be decoded (for example, u0, v0, sizeU0, and sizeV0), three-dimensional information (for example, u1, d1, and v1), and rotation transformation information ( For example, patch[k].orientationIndex), matching patch index (such as patch[k].matchedPatchIndex), and even the global matching patch identifier of the patch in the first frame of the point cloud in the subgroup (such as patch[k].globalPatchFlag), etc.;
  • the auxiliary information decoding module 204 decodes to obtain side information, such as three-dimensional information of the patch to be decoded (for example, u1, d1, and v1), and matching patch index (for example, patch[k].matchedPatchIndex).
  • the auxiliary information decoding module 204 can use different decoding methods (for example, fixed-length decoding or differential decoding) to pars
  • the geometric information reconstruction module 205 of the point cloud is configured to reconstruct the geometric information of the point cloud (for example, reconstruct the patch in the point cloud) according to the received occupancy map information, depth map information and auxiliary information. After the geometric information of the reconstructed point cloud is filtered by the point cloud filtering module 206, it is sent to the point cloud texture information reconstruction module 207.
  • the decoder 200 shown in FIG. 4 is only an example. In a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 4. The embodiment of the present application does not limit this.
  • the encoder first divides the point cloud to be coded (that is, the current frame or the current frame point cloud) into a number of patches according to certain criteria, and these patches have no overlapping areas between them. Then, project each patch from the three-dimensional space to the two-dimensional plane to obtain a two-dimensional image (that is, the occupancy map of the patch). Then, the occupancy maps of all patches (or the occupancy maps of the patches after reduced resolution) are tightly arranged on a two-dimensional image according to a certain rule to obtain the current frame occupancy map. This method of arranging patch occupancy maps is called packing. Subsequently, the current frame depth map and the current frame texture map are generated according to the packing order.
  • the current frame depth map is a two-dimensional image generated by the depth obtained by each patch through the packing order.
  • the texture map of the current frame is a two-dimensional image generated by the texture map obtained by projection of each patch according to the packing order.
  • the current frame occupancy map is a two-value two-dimensional image used to indicate whether each pixel position of the two-dimensional image is occupied by a point in the point cloud.
  • the resolution of the current frame occupancy map is lower than the resolution of the current frame depth map and the current frame texture map.
  • the point cloud group to be coded includes a multi-frame point cloud, and the multi-frame point cloud may be a time-continuous or non-continuous multi-frame point cloud.
  • the point cloud group to be coded may be one or more frame groups (group of frame, GOF), and one GOF may include 32 frame point clouds, which is of course not limited thereto.
  • the point clouds in a point cloud group are divided into several subgroups, and each subgroup includes one or more frames of point clouds.
  • a point cloud group (including 32-frame point clouds) is divided into multiple sub-groups.
  • the 0th subgroup includes the 0th, 1, 2, and 3th frame point clouds in the point cloud group
  • the 1st subgroup includes the 4th and 5th frame point clouds in the point cloud group
  • the second subgroup includes the point cloud group.
  • the third subgroup includes the point cloud at frames 11, 12, 13, 14, and 15 in the point cloud group.
  • any sub-group into which the point cloud to be coded is divided may contain one or more frames of point cloud.
  • the number of point clouds contained in different subgroups may be equal or unequal.
  • the method of dividing the point cloud group to be coded into multiple subgroups is not limited in this application.
  • the matching patch in this application refers to a patch that finds a matching patch (also called a reference patch) in the reference frame of the point cloud to which the patch belongs.
  • Non-matching patch refers to the patch that does not find a matching patch in the reference frame of the point cloud to which the patch belongs.
  • the specific method of finding a matching patch is not limited in this application.
  • the point cloud to which the matching patch belongs and its reference frame can belong to the same subgroup (for example, the reference patch of the patch in the non-first frame point cloud in the second subgroup can be the patch in the same subgroup), or it can belong to different
  • the reference patch of the patch in the point cloud of the first frame (ie, the sixth frame) in the second subgroup can be the patch of the point cloud frame (tail frame) in the zeroth subgroup or the first subgroup).
  • a global matching patch within a subgroup refers to a patch that has a matching relationship across all point clouds in a subgroup (also referred to as within a subgroup). For example, suppose a subgroup has 4 points of point clouds (respectively labeled as point clouds 1 to 4), for any patch in point cloud 1, if it can be found in point clouds 2 to 4 that matches the patch
  • the patch of the relationship for example, the patches that have a matching relationship with patch11 in point cloud 1 are: patch21 in point cloud 2, patch31 in point cloud 3, and patch41 in point cloud 4.
  • patch11, 21, 31, 41 Both are the global matching patches in this subgroup
  • patchq1 is the global matching patch in the current point cloud frame q, 1 ⁇ q ⁇ 4, and q is an integer.
  • a non-global matching patch in a subgroup refers to a patch that has a matching relationship across some point clouds in a subgroup (also known as a subgroup), or a non-global matching patch is all point clouds in the subgroup No matching patch was found in.
  • the side information of the patch to be coded is encoded into the code stream, including: if the patch to be coded is the first frame point in the subgroup The patch in the cloud, and the patch to be coded is a global matching patch and the patch to be coded has a matching patch.
  • the side information of the patch to be coded is compiled into the code stream, and the matching patch index of the patch to be coded is a non-default value ,
  • the global matching patch index is the first value.
  • the side information of the patch to be coded is compiled into the code stream, including: if the patch to be coded is a patch in the point cloud of the non-first frame in the subgroup, and
  • the patch to be encoded is a non-globally matching patch and the patch to be encoded does not have a matching patch.
  • the side information of the patch to be encoded also includes two-dimensional information, normal coordinate axis information and rotation transformation information.
  • the side information of the patch to be encoded Program into the code stream.
  • the three-dimensional information of the patch to be decoded is parsed from the code stream by means of differential decoding.
  • the normal coordinate axis information of the patch to be decoded is obtained according to the matching patch index of the patch to be decoded , Including: if the matching patch index of the patch to be decoded is the default value, the normal axis information of the patch to be decoded is also parsed from the code stream.
  • the patch to be decoded The difference between the v0 of the patch to be decoded and the v0 of the reference patch, the difference between the sizeU0 of the patch to be decoded and the sizeU0 of the reference patch, and the difference between the sizeV0 of the patch to be decoded and the sizeV0 of the reference patch; then according to The difference between the u0 of the patch to be decoded and the u0 of the reference patch and the u0 of the reference patch obtain the u0 of the patch to be decoded, and the difference between the v0 of the patch to be decoded and the v0 of the reference patch and the v0 of the reference patch are obtained.
  • the partial side information of the global matching patch in the subgroup is decoded separately, and the partial side information refers to the side information that is shared by the global matching patch and has the same value.
  • the partial side information refers to part or all of the two-dimensional information, normal coordinate axis information, and rotation transformation information.
  • S703 Reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.
  • the reference frame of the point cloud of the 0th frame of the I-th subgroup is the last frame of the point cloud of the I-1th subgroup.
  • the first subgroup includes 3 frames of point clouds, which are respectively the 0th frame point cloud, the 1st frame point cloud and the 2nd frame point cloud; each frame of point cloud includes 9 patches.
  • the reference frame of the 0th frame point cloud in the I-th subgroup includes matching with the 0th, 1, 2, 3, 4, and 5th patches in the 0th frame of the point cloud in the I-th subgroup. Patches that do not match the 6, 7, and 8 patches of the 0th frame point cloud in the Ith subgroup.
  • the reference frame of the point cloud of the second frame in the first subgroup is the point cloud of the first frame in the first subgroup.
  • the reference frames that match the 0th, 1, 2, 3, 4, and 5th patches in the point cloud of the 1st frame in the I-th subgroup respectively patch, and a patch that does not match the 6, 7, and 8 patches in the second frame point cloud in the I subgroup.
  • the 0th, 1, 2, 3, 4, and 5th patches of the second frame point cloud in the I-th subgroup are all matching patches, and the 6, 7 and 8th patches are non-matching patches.
  • the 0th, 1, 2, 3, 4, and 6th patches of the 0th frame point cloud in the I-th subgroup, the 0th, 1, 2, 3, 4, and 6th patches and the first frame point cloud The 0th, 1, 2, 3, 4, and 5th patches in the 2 frame point cloud are all global matching patches; the 5th, 7th, 8th patch and the 1st frame point of the 0th frame point cloud in the I subgroup
  • the 5th, 7th, and 8th patches in the cloud and the 6th, 7th, and 8th patches in the second frame point cloud are all non-global matching patches.
  • the 0,1,2,3,4,6 patches in the point cloud of the 1st frame are all global matching patches, and the 0,1,
  • the three-dimensional information and matching patch index of each patch in 2,3,4,6 patches, and the three-dimensional information and matching patch index of each patch in the 0,1,2,3,4,6th patch are compiled Code stream.
  • the difference between the three-dimensional information of each patch in the 0th, 1, 2, 3, 4, and 6th patches and the 3D information of the reference patch is encoded into the code stream by means of differential coding.
  • the 5th and 7th patches in the point cloud of the first frame are all matching patches and are non-global matching patches.
  • the eighth patch in the point cloud of the first frame is a non-matching patch.
  • the two-dimensional information, three-dimensional information, normal axis information, rotation transformation information and matching patch index of the eighth patch in the point cloud of the first frame are compiled into the code stream.
  • the matching patch index of each patch it is determined that the 0th, 1, 2, 3, 4, and 5th patches are all matching patches, and the 6th, 7th, and 8th patachs are all non-matching patches.
  • the normal axis information of the reference patch of each of the 2, 3, 4, and 5 patches determines the normal axis information of each of the 0, 1, 2, 3, 4, and 5th patches; and from Analyze the normal axis information of each patch in the sixth, seventh, and eighth patches in the code stream.
  • each patch of the 0th, 1, 2, 3, 4, 5, 6, and 7 patches determine that the 0th, 1, 2, 3, 4, and 6th patches are all global matching patches, and determine the first 5,7 patches are non-global matching patches.
  • the identifier is a second value (such as 0 or false), and the second value is used to indicate that the patch is a non-global matching patch.
  • an embodiment of the present application also provides a point cloud encoding method, including: obtaining side information of the patch to be encoded; and encoding instruction information into a code stream, and the instruction information is used to indicate whether to follow the target encoding method
  • a point cloud encoding method including: obtaining side information of the patch to be encoded; and encoding instruction information into a code stream, and the instruction information is used to indicate whether to follow the target encoding method
  • the embodiment of the present application further provides a point cloud decoding method, including: parsing a code stream to obtain indication information, the indication information is used to indicate whether to decode side information according to the target decoding method; when the indication information It is used to indicate that when the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; see part or all of the content of the embodiment shown in FIG. 7 for the target decoding method, which will not be described here; The side information of the patch reconstructs the patch to be decoded.
  • an embodiment of the present application also provides a point cloud encoding method, including: obtaining side information of the patch to be encoded; and encoding instruction information into a code stream, and the instruction information is used to indicate whether to follow the target encoding method
  • a point cloud encoding method including: obtaining side information of the patch to be encoded; and encoding instruction information into a code stream, and the instruction information is used to indicate whether to follow the target encoding method
  • the embodiment of the present application further provides a point cloud decoding method, including: parsing a code stream to obtain indication information, the indication information is used to indicate whether to decode side information according to the target decoding method; when the indication information It is used to indicate that when the side information is decoded according to the target decoding method, the side information of the patch to be decoded is obtained according to the target decoding method; see part or all of the content of the embodiment shown in FIG. 7 for the target decoding method, which will not be described here; The side information of the patch reconstructs the patch to be decoded.
  • FIG. 9 is a schematic structural diagram of a point cloud encoding device provided by an embodiment of the application.
  • the encoding device 900 may include:
  • the auxiliary information encoding module 902 is used to encode the side information of the patch to be encoded into the code stream if the patch to be encoded is the patch in the first frame of the point cloud in the subgroup;
  • the side information includes 2D information, 3D information, and rotation transformation Information, global matching patch identification and matching patch index;
  • the global matching patch identification is used to indicate whether the patch to be encoded is a global matching patch in the subgroup;
  • the auxiliary information encoding module 902 is also used to encode the side information of the patch to be encoded into the code stream if the patch to be encoded is a patch in the point cloud of the non-first frame in the subgroup.
  • the side information includes three-dimensional information and a matching patch index.
  • the auxiliary information encoding module 902 is specifically configured to:
  • the patch to be coded is the patch in the point cloud of the first frame in the subgroup, and the patch to be coded is a global matching patch and the patch to be coded has a matching patch, and the side information of the patch to be coded is encoded into the code stream.
  • the auxiliary information encoding module 902 is specifically configured to: if the patch to be encoded is in the subgroup The patch in the point cloud of the non-first frame, and the patch to be encoded is a global matching patch and the patch to be encoded has a matching patch, and the side information of the patch to be encoded is encoded into the code stream.
  • the matching patch index of the patch to be encoded is a non-default value to indicate a reference patch that matches the patch to be encoded.
  • the global matching patch identifier of the patch to be encoded is the first value to indicate that the patch to be encoded is a global matching patch in the subgroup.
  • the auxiliary information encoding module 902 is specifically configured to:
  • the patch to be coded is the patch in the point cloud of the first frame in the subgroup, and the patch to be coded is a global matching patch and the patch to be coded does not have a matching patch.
  • the side information of the patch to be coded also includes the normal axis information.
  • the side information of the patch to be encoded is encoded into the code stream; wherein the matching patch index is used to indicate that the patch to be encoded does not have a matching patch.
  • the matching patch index of the patch to be coded is the default value to indicate that the patch to be coded has no matching patch;
  • the global matching patch identifier of the patch to be coded is the first value to indicate that the patch to be coded is a global match in the subgroup patch.
  • the auxiliary information encoding module 902 is specifically configured to:
  • the patch to be coded is the patch in the point cloud of the first frame in the subgroup, and the patch to be coded is a non-globally matched patch and the patch to be coded has a matching patch, and the side information of the patch to be coded is encoded into the code stream;
  • the auxiliary information encoding module 902 is specifically configured to: if the patch to be encoded is in the subgroup The patch in the non-first frame point cloud, and the patch to be coded is a non-global matching patch and the patch to be coded has a matching patch.
  • the side information of the patch to be coded also includes two-dimensional information. Edit the side information of the patch to be coded Into the code stream.
  • the matching patch index of the patch to be encoded is a non-default value to indicate the reference patch that matches the patch to be encoded;
  • the global matching patch identifier of the patch to be encoded is the second value to indicate that the patch to be encoded is not in the subgroup Match patch globally.
  • the auxiliary information encoding module 902 is specifically configured to:
  • the patch to be coded is the patch in the point cloud of the first frame in the subgroup, and the patch to be coded is a non-globally matching patch and the patch to be coded does not have a matching patch.
  • the side information of the patch to be coded also includes normal axis information, Coding the side information of the patch to be encoded into the code stream;
  • the auxiliary information encoding module 902 is specifically configured to: if the patch to be encoded is in the subgroup The patch in the non-first frame point cloud, and the patch to be coded is a non-global matching patch and the patch to be coded does not have a matching patch.
  • the side information of the patch to be coded also includes two-dimensional information, normal coordinate axis information and rotation Transform the information, and encode the side information of the patch to be encoded into the code stream.
  • the matching patch index of the patch to be coded is the default value to indicate that the patch to be coded has no matching patch;
  • the global matching patch identifier of the patch to be coded is the second value to indicate that the patch to be coded is not in the subgroup Match patch globally.
  • the patch information acquisition module 901 is used to execute the related content of step S601 in the embodiment shown in FIG. 6, and the auxiliary information encoding module 902 is used to execute the related content of steps S602 and S603 in the embodiment shown in FIG. 6.
  • the patch information acquisition module 901 may correspond to the combination of the patch information generation module 101 and the packing module 102 in FIG. 2
  • the auxiliary information encoding module 902 may correspond to the auxiliary information encoding module 108 in FIG. 2.
  • the patch information The function of the acquiring module 901 may be implemented by the patch information generating module 101 and the packing module 102 in FIG.
  • the function of the auxiliary information generating module 902 may be implemented by the auxiliary information encoding module 108 in FIG. 2.
  • the patch information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and global matching patch identifiers of the patch to be encoded, and sizeU0 and sizeV0 in the two-dimensional information
  • the packaging module 102 is used to obtain the patch to be encoded
  • the rotation transformation information, matching patch index and u0 and v0 in the two-dimensional information is not limited to this.
  • the decoding device 1000 includes:
  • the auxiliary information decoding module 1001 is used to parse the side information of the patch to be decoded from the code stream;
  • the side information includes 2D information, 3D information, rotation transformation information, global matching patch identification and matching patch index
  • the normal coordinate axis information of the patch to be decoded is obtained according to the matching patch index
  • the side information includes three-dimensional information and a matching patch index, obtain the global matching patch identifier, two-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded according to the matching patch index of the patch to be decoded;
  • the reconstruction module 1002 is used to reconstruct the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded; wherein the global matching patch identifier is used to indicate whether the patch to be decoded is a global matching patch.
  • the auxiliary information decoding module 1001 is specifically used for:
  • the matching patch index of the patch to be decoded is a non-default value, then the normal coordinate axis information of the reference patch indicated by the matching patch index is determined.
  • the auxiliary information decoding module 1001 is specifically used to: if it is determined that the matching patch index of the patch to be decoded is a non-default value, the global matching patch identifier of the reference patch is used to determine the global matching patch identifier of the patch to be decoded; if the global matching patch identifier of the patch to be decoded is the first A value is used to determine the two-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded according to the reference patch's two-dimensional information, normal coordinate axis information, and rotation transformation information.
  • the auxiliary information decoding module 1001 is specifically used for:
  • the axis information of the patch to be decoded is also parsed from the code stream.
  • the auxiliary information decoding module 1001 is specifically used for:
  • the normal coordinate axis information of the reference patch indicated by the matching patch index is determined according to the normal coordinate axis information of the patch to be decoded;
  • the auxiliary information decoding module 1001 is specifically used for:
  • the global matching patch identifier of the reference patch indicated by the matching patch index is determined to determine the global matching patch identifier of the patch to be decoded; if the global matching patch identifier of the patch to be decoded is determined Is the second value; then the two-dimensional information of the patch to be decoded is also parsed from the code stream; and the normal coordinate axis of the patch to be decoded is determined according to the normal coordinate axis information and the rotation transformation information of the reference patch indicated by the matching patch index Information and rotation transformation information.
  • the auxiliary information decoding module 1001 is specifically used for:
  • the normal coordinate axis information of the patch to be decoded is also parsed from the code stream;
  • the auxiliary information decoding module 1001 is specifically used to: if it is determined that the matching patch index of the patch to be decoded is the default value, then the two-dimensional information, normal coordinate axis information and rotation transformation information are also analyzed from the code stream, and the global matching patch identifier of the patch to be decoded is determined Is the second value.
  • the auxiliary information decoding module 1001 is used to execute the related content of steps S701 and S702 in the embodiment shown in FIG. 7, and the reconstruction module 1002 is used to execute the related content of step S703 in the embodiment shown in FIG. 7.
  • the auxiliary information decoding module 1001 corresponds to the auxiliary information decoding module 204 in FIG. 4
  • the reconstruction module 1002 corresponds to the geometric information reconstruction module 205 of the point cloud in FIG. 4.
  • the function of the auxiliary information decoding module 1001 It may be implemented by the auxiliary information decoding module 204 in FIG. 4, and the function of the reconstruction module 1002 may be implemented by the geometric information reconstruction module 205 of the point cloud in FIG.
  • this application is not limited to this.
  • FIG. 11 is a schematic structural diagram of a point cloud encoding device provided by an embodiment of the application.
  • the encoding device 1100 includes:
  • the patch information obtaining module 1101 is configured to obtain side information of the patch to be encoded
  • the auxiliary information decoding module 1201 in FIG. 12 corresponds to the auxiliary information decoding module 204 in FIG. 4, and the reconstruction module 1202 in FIG. 12 corresponds to the geometric information reconstruction of the point cloud in FIG.
  • the function of the auxiliary information decoding module 1201 can be realized by the auxiliary information decoding module 204 in FIG. 4
  • the function of the reconstruction module 1202 can be realized by the point cloud geometric information reconstruction module 205 in FIG. 4 .
  • this application is not limited to this.
  • the encoding apparatus 1300 may further include one or more output devices, such as a display 1304.
  • the display 1304 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input.
  • the display 1304 may be connected to the processor 1301 via the bus 1303.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is properly termed a computer-readable medium.
  • any connection is properly termed a computer-readable medium.
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave
  • coaxial cable Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media.
  • magnetic disks and optical disks include compact disks (CD), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
  • DSP digital signal processors
  • ASICs application-specific integrated circuits
  • FPGA field programmable logic arrays
  • the term "processor” as used herein may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein.
  • the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec.
  • the technology can be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.
  • the technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets).
  • ICs integrated circuits
  • a set of ICs for example, chipsets.
  • Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units.
  • various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.

Abstract

L'invention concerne un procédé de codage/décodage de nuage de points et un codeur/décodeur, se rapportant au domaine technique du codage/décodage, et permettant d'améliorer l'efficacité de codage/décodage de données de nuage de points et de réduire le surdébit de bits d'un flux de code. Le procédé de codage de nuage de points consiste à : acquérir des informations annexes d'un correctif à coder ; si le correctif à coder est un correctif d'un nuage de points de la première trame d'un sous-groupe, coder les informations annexes du correctif à coder en un flux de code, les informations annexes comprenant des informations bidimensionnelles, des informations tridimensionnelles, des informations de transformation de rotation, un identifiant de correctif d'appariement global et un index de correctif d'appariement, l'identifiant de correctif d'appariement global servant à indiquer si le correctif à coder est un correctif d'appariement global du sous-groupe ; et si le correctif à coder est un correctif d'un nuage de points d'une trame autre que la première trame du sous-groupe, coder les informations annexes du correctif à coder en le flux de code, les informations annexes comprenant les informations tridimensionnelles et l'index de correctif d'appariement.
PCT/CN2020/071022 2019-01-23 2020-01-08 Procédé et appareil de codage/décodage de nuage de points WO2020151496A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910070741.XA CN111479114B (zh) 2019-01-23 2019-01-23 点云的编解码方法及装置
CN201910070741.X 2019-01-23

Publications (1)

Publication Number Publication Date
WO2020151496A1 true WO2020151496A1 (fr) 2020-07-30

Family

ID=71736101

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/071022 WO2020151496A1 (fr) 2019-01-23 2020-01-08 Procédé et appareil de codage/décodage de nuage de points

Country Status (2)

Country Link
CN (1) CN111479114B (fr)
WO (1) WO2020151496A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114667728A (zh) * 2020-12-31 2022-06-24 深圳市大疆创新科技有限公司 点云编解码方法、装置及系统
WO2023050951A1 (fr) * 2021-09-30 2023-04-06 Beijing Xiaomi Mobile Software Co., Ltd. Procédé et appareil de codage/décodage de données de géométrie de nuage de points détectées par au moins un capteur

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111866484B (zh) * 2019-04-30 2023-06-20 华为技术有限公司 点云编码方法、点云解码方法、装置及存储介质
CN114078191A (zh) * 2020-08-18 2022-02-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法、装置、设备及介质
CN116781674A (zh) * 2022-03-08 2023-09-19 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN107633539A (zh) * 2017-09-25 2018-01-26 潍坊学院 一种基于四边面片分割的三维点云模型数据压缩方法
CN108632607A (zh) * 2018-05-09 2018-10-09 北京大学深圳研究生院 一种基于多角度自适应帧内预测的点云属性压缩方法
CN108833927A (zh) * 2018-05-03 2018-11-16 北京大学深圳研究生院 一种基于删除量化矩阵中0元素的点云属性压缩方法
CN109196559A (zh) * 2016-05-28 2019-01-11 微软技术许可有限责任公司 动态体素化点云的运动补偿压缩

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005083684A1 (fr) * 2004-02-19 2005-09-09 Koninklijke Philips Electronics N.V. Schema de decodage pour signaux a longueurs d'unites variables
WO2005091632A1 (fr) * 2004-02-23 2005-09-29 Nokia Corporation Envoi d'informations video
KR101700966B1 (ko) * 2008-01-10 2017-01-31 톰슨 라이센싱 인트라 예측 비디오의 조명 보상 방법 및 장치
EP2904780A1 (fr) * 2012-12-18 2015-08-12 Siemens Aktiengesellschaft Procédé pour le codage d'une séquence d'images numériques
CN105430406B (zh) * 2015-12-04 2018-06-12 中国矿业大学 一种分布式视频编码框架
CN106157371A (zh) * 2016-07-11 2016-11-23 南京理工大学 基于自适应周向扩张策略的散乱点云高效网格化方法
US10832471B2 (en) * 2017-03-27 2020-11-10 Mitsubishi Electric Research Laboratories, Inc. Fast T-spline fitting system and method
CN108053446A (zh) * 2017-12-11 2018-05-18 北京奇虎科技有限公司 基于点云的定位方法、装置及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170214943A1 (en) * 2016-01-22 2017-07-27 Mitsubishi Electric Research Laboratories, Inc. Point Cloud Compression using Prediction and Shape-Adaptive Transforms
CN109196559A (zh) * 2016-05-28 2019-01-11 微软技术许可有限责任公司 动态体素化点云的运动补偿压缩
CN107633539A (zh) * 2017-09-25 2018-01-26 潍坊学院 一种基于四边面片分割的三维点云模型数据压缩方法
CN108833927A (zh) * 2018-05-03 2018-11-16 北京大学深圳研究生院 一种基于删除量化矩阵中0元素的点云属性压缩方法
CN108632607A (zh) * 2018-05-09 2018-10-09 北京大学深圳研究生院 一种基于多角度自适应帧内预测的点云属性压缩方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114667728A (zh) * 2020-12-31 2022-06-24 深圳市大疆创新科技有限公司 点云编解码方法、装置及系统
CN114667728B (zh) * 2020-12-31 2023-10-13 深圳市大疆创新科技有限公司 点云编解码方法、装置及系统
WO2023050951A1 (fr) * 2021-09-30 2023-04-06 Beijing Xiaomi Mobile Software Co., Ltd. Procédé et appareil de codage/décodage de données de géométrie de nuage de points détectées par au moins un capteur

Also Published As

Publication number Publication date
CN111479114A (zh) 2020-07-31
CN111479114B (zh) 2022-07-22

Similar Documents

Publication Publication Date Title
WO2020151496A1 (fr) Procédé et appareil de codage/décodage de nuage de points
US20210029381A1 (en) Method and apparatus for obtaining global matched patch
CN110662087B (zh) 点云编解码方法和编解码器
CN110971912B (zh) 点云编解码方法、编解码器、编解码装置和存储介质
US11961265B2 (en) Point cloud encoding and decoding method and apparatus
CN110944187B (zh) 点云编码方法和编码器
WO2020011265A1 (fr) Procédé de codage et de décodage de nuage de points et codeur-décodeur
WO2020147379A1 (fr) Procédé et dispositif de filtrage de nuage de points et support de stockage
US20220007037A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
CN111726615B (zh) 点云编解码方法及编解码器
WO2020143725A1 (fr) Procédé de décodage de nuage de points et décodeur
WO2020187283A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points, appareil et support de stockage
WO2020220941A1 (fr) Procédé d'encodage de nuage de points, procédé de décodage de nuage de points, appareils et support de stockage
JP2022513484A (ja) 点群符号化方法及びエンコーダ
WO2020015517A1 (fr) Procédé de codage de nuage de points, procédé de décodage de nuage de points et décodeur
WO2020057338A1 (fr) Procédé de codage en nuage de points et codeur
CN118042135A (en) Point cloud encoding method, point cloud decoding method, device and storage medium
WO2020187191A1 (fr) Procédé de codage et de décodage de nuage de points et codec

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20744537

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20744537

Country of ref document: EP

Kind code of ref document: A1