WO2020063246A1 - 点云编解码方法和编解码器 - Google Patents

点云编解码方法和编解码器 Download PDF

Info

Publication number
WO2020063246A1
WO2020063246A1 PCT/CN2019/103293 CN2019103293W WO2020063246A1 WO 2020063246 A1 WO2020063246 A1 WO 2020063246A1 CN 2019103293 W CN2019103293 W CN 2019103293W WO 2020063246 A1 WO2020063246 A1 WO 2020063246A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
union
patches
auxiliary information
map
Prior art date
Application number
PCT/CN2019/103293
Other languages
English (en)
French (fr)
Inventor
张德军
王田
扎克哈成科·弗莱德斯拉夫
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201980060952.3A priority Critical patent/CN112740700A/zh
Priority to EP19866951.7A priority patent/EP3852373A4/en
Publication of WO2020063246A1 publication Critical patent/WO2020063246A1/zh
Priority to US17/217,404 priority patent/US11388442B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/004—Predictors, e.g. intraframe, interframe coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Definitions

  • the present application relates to the field of codec technology, and in particular, to a point cloud codec method and a codec.
  • 3d sensor such as 3d scanner
  • the embodiments of the present application provide a point cloud encoding and decoding method and a codec, which are used to provide a technical solution for efficiently encoding and decoding point cloud data. Specifically, it helps to save code without affecting the encoding and decoding performance. Streaming overhead.
  • a point cloud encoding method including: obtaining auxiliary information of a union set occupation map of a point cloud group; wherein the point cloud group includes N frame point clouds, N ⁇ 2, N is an integer, and the union set
  • the occupancy map is the union of the occupancy maps of the N point patches with matching relationships in the N-frame point cloud; the auxiliary information of the occupancy map is used to determine (specifically, used by the decoder to determine) the Auxiliary information of N patches;
  • a syntax element is coded into the code stream, and the syntax element includes auxiliary information of the union occupation map.
  • the execution subject of this technical solution may be an encoder.
  • the patch that has a matching relationship with a patch is the matching patch of the target patch, and the target patch is the matching patch of the patch.
  • Each of the N patches can be referred to as a global matched patch (global matched patch).
  • the position information of the union occupancy map in the global occupancy map corresponds to the occupancy map of the above N patches
  • the position information in the occupancy map of the point cloud to which it belongs, and / or the union occupancy map is the union of the occupancy maps of the N patches, etc. Therefore, the N occupants can be represented or approximated by the auxiliary information of the union occupancy map. Auxiliary information for patch. In this way, the code stream transmission overhead can be saved without affecting the performance of the codec.
  • the coordinate values of the union occupation map in the global occupation map include: u0 and / or v0 of the union occupation map.
  • the coordinate value corresponding to the union occupation map in the three-dimensional space refers to the minimum (or maximum) coordinate value of the N patches corresponding to the union occupation map in the three-dimensional space.
  • the coordinate value corresponding to the three-dimensional space of the union occupation map may include: the minimum u1 value, the minimum v1 value, and / or the minimum d1 value of the N patches in the three-dimensional space.
  • the corresponding auxiliary information of the one or more patches refers to auxiliary information of the same type as the auxiliary information of the union occupancy graph, for example, they are all u0 and the like.
  • This possible design uses a differential method to transmit the auxiliary information of one or more patches corresponding to the occupancy map. In this way, it helps to improve the accuracy and / or setting of the auxiliary information of the one or more patches in the N patches. flexibility.
  • acquiring auxiliary information of the union occupancy map of the point cloud group includes: acquiring auxiliary information of the union occupancy map based on the auxiliary information of the N patches.
  • acquiring the auxiliary information of the union occupation map based on the auxiliary information of the N patches includes:
  • the auxiliary information of the union occupancy map includes the coordinate values of the union occupancy map in the global occupancy map, the maximum or minimum coordinate values of the occupancy map of the N patches in the occupancy map of the corresponding point cloud, As the coordinate value of the union occupation map in the global occupation map.
  • the minimum or maximum value of u0 of the occupation map of the N patches is used as u0 of the occupation map of the union; and / or the minimum value or maximum value of v0 of the occupation map of the N patches is used as the union. Occupies the v0 of the graph.
  • the maximum coordinate value or the minimum coordinate value of the N patches in the corresponding point cloud is used as the union
  • the set occupancy map corresponds to coordinate values in three-dimensional space. For example, taking the minimum coordinate value as an example, the minimum value of u1 of the N patches is used as the value of u1 of the union occupation graph, and the minimum value of v1 of the N patches is used as the v1 of the union occupation graph. The minimum value of d1 of the N patches is used as the value of d1 of the union occupation graph.
  • the coordinate axis of an occupation map of a patch in the occupation map of the corresponding point cloud is the coordinate value of the occupation map of the patch in the occupation map of the point cloud to which the patch belongs.
  • the coordinate value of a patch in the corresponding point cloud is the coordinate value of the patch in the point cloud to which the patch belongs.
  • the point cloud group is a group of frames (GOF) GOF or a sub-GOF
  • the sub-GOF includes a continuous multi-frame point cloud in one GOF.
  • the above syntax element further includes information indicating a position of a point cloud (such as a point cloud per frame) contained in the sub-GOF in the GOF to which the sub-GOF belongs, For example, the number of the start frame of the sub-GOF and / or the number of frames included in the sub-GOF.
  • a point cloud decoding method including: parsing a bitstream to obtain syntax elements, the syntax elements including auxiliary information of a union set occupation map of a point cloud group, the point cloud group including N frame point clouds, N ⁇ 2, N is an integer, the union occupancy map is the union of the occupancy maps of N patches with matching relationship in the N-frame point cloud; the N patches are determined according to the auxiliary information of the union occupancy map obtained through analysis Based on auxiliary information of multiple patches in the N-frame point cloud, reconstructing the N-frame point cloud; wherein the plurality of patches include the N patches.
  • the execution subject of this technical solution may be a decoder.
  • the method may further include displaying the reconstructed point cloud.
  • the auxiliary information of the union occupation map includes at least one of the following information: the coordinate values of the union occupation map in the global occupation map, the size information of the union occupation map in the global occupation map, and The coordinate values of the union occupancy map in three-dimensional space.
  • the global occupancy map is an occupancy map obtained by packing multiple union occupancy maps in a point cloud group.
  • the syntax element further includes: a difference between corresponding auxiliary information of one or more patches in the N patches and auxiliary information of the union occupation map.
  • the auxiliary information of the N patches is determined according to the auxiliary information of the union occupation map obtained by analysis, including: the auxiliary information of the union occupation map obtained according to the analysis, and one of the N patches or The difference between the corresponding auxiliary information of multiple patches and the auxiliary information of the union occupancy graph determines the auxiliary information of the N patches.
  • the auxiliary information of the union occupation map and the difference between the corresponding auxiliary information of one of the N patches and the auxiliary information of the union occupation map are summed to obtain the corresponding auxiliary information of the patch.
  • the point cloud group is a GOF or a sub-GOF.
  • the sub-GOF includes a continuous multi-frame point cloud in a GOF.
  • the above syntax element further includes: information indicating the position of each point cloud included in the sub-GOF in the GOF to which the sub-GOF belongs, such as the start of the sub-GOF The number of the start frame and / or the number of frames included in the sub-GOF.
  • the above method further includes: determining, according to the parsed information indicating the position of each frame of point cloud included in the sub-GOF, the GOF to which the sub-GOF belongs, determining the frame of point cloud included in the sub-GOF in the sub-GOF Position in the GOF to which it belongs.
  • the second aspect or any possible design provided by the second aspect corresponds to the first aspect or the possible design corresponding to the first aspect. Therefore, the second aspect or any one of the possible designs provided by the second aspect.
  • the explanation of the related content, the specific implementation manner, and the description of the beneficial effects can be obtained by referring to the description in the first aspect or the possible design of the first aspect, or inferred from the description in the first aspect or the possible design of the first aspect. , Will not repeat them here.
  • a point cloud encoding method which includes: determining instruction information, and coding the instruction information into a code stream.
  • the indication information indicates whether the target point cloud encoding method is used to encode the auxiliary information of the N patches in the point cloud group, and the target point cloud encoding method includes the foregoing first aspect or any one of the possible designs provided by the first aspect.
  • the described point cloud encoding method includes N frame point clouds, N ⁇ 2, and N is an integer; the N patches are N patches having a matching relationship in the N frame point cloud. That is, the technical solution provided in the embodiment of the present application supports the technical solution of selecting a target point cloud coding method from at least two candidate point cloud coding methods.
  • a point cloud decoding method including: parsing a bitstream to obtain instruction information indicating whether to use a target point cloud decoding method to decode auxiliary information of N patches in a point cloud group.
  • the target point cloud decoding method includes the point cloud decoding method described in the second aspect or any one of the possible designs provided in the second aspect; the point cloud group includes N frame point clouds, N ⁇ 2, and N is an integer; the N A patch is N patches with a matching relationship in the N-frame point cloud; when the instruction information indicates that the target point cloud decoding method is used for decoding, the target point cloud decoding method is used to process the auxiliary information of the N patches. That is, the technical solution provided in the embodiment of the present application supports a technical solution of selecting a target point cloud decoding method from at least two candidate point cloud decoding methods.
  • an encoder including: each module for performing the foregoing first aspect or the method provided by any possible design of the first aspect, or for performing the foregoing third aspect or any one of the third aspect The various modules of the method provided by a possible design.
  • the encoder includes a packaging module and an auxiliary information encoding module.
  • the packaging module is used to obtain auxiliary information of the union occupancy map of the point cloud group;
  • the point cloud group includes N frame point clouds, N ⁇ 2, N is an integer, and the
  • the set occupancy map is the union of the occupancy maps of the N patches with matching relationships in the N-frame point cloud;
  • the auxiliary information of the occupancy map of the set is used to determine the auxiliary information of the N patches.
  • the auxiliary information encoding module is used to encode a syntax element containing auxiliary information of the union occupation map into a code stream.
  • the packaging module is configured to determine instruction information that indicates whether to use the target point cloud encoding method to encode the auxiliary information of N patches in the point cloud group.
  • the point cloud encoding method includes the point cloud encoding method described in the first aspect or any one of the possible designs provided by the first aspect; the point cloud group includes N frame point clouds, N ⁇ 2, and N is an integer; the N patch is N patches with matching relationship in the N-frame point cloud.
  • the auxiliary information encoding module is configured to encode the instruction information into a code stream.
  • a decoder including: each module for implementing the second aspect or the method provided by any possible design of the second aspect, or for implementing the fourth aspect or any one of the fourth aspect.
  • the decoder may be divided into functional modules according to the method provided in the second / fourth aspect or any one of the possible implementation manners in the second / fourth aspect, for example, corresponding to each function. Dividing each functional module, you can also integrate two or more functions into one processing module.
  • the decoder includes an auxiliary information decoding module and a point cloud reconstruction module.
  • the auxiliary information decoding module is used to parse a code stream to obtain a syntax element, and the syntax element includes auxiliary information of a union occupation map of the point cloud group, and the union occupation map Is the union of the occupancy maps of N patches with matching relationships in the N-frame point cloud; the auxiliary information of the N patches is determined according to the auxiliary information of the occupancy map of the union set obtained by analysis.
  • the point cloud reconstruction module is configured to reconstruct the N-frame point cloud based on auxiliary information of multiple patches in the N-frame point cloud, where the multiple patches include the N patches.
  • the auxiliary information decoding module is used to parse the code stream to obtain instruction information that indicates whether to use the target point cloud decoding method for the N patches in the point cloud group.
  • the auxiliary information is decoded, and the target point cloud decoding method includes the point cloud decoding method described in the second aspect or any one of the possible designs provided in the second aspect;
  • the point cloud group includes N frame point clouds, N ⁇ 2, N Is an integer;
  • the N patches are N patches with a matching relationship in the N-frame point cloud; and when the instruction information indicates that the target point cloud decoding method is used for decoding, the N point patches are used for the N patches Decode the auxiliary information.
  • the point cloud reconstruction module is configured to reconstruct the N frame point cloud based on the auxiliary information of the plurality of patches in the N frame point cloud obtained by decoding; the plurality of patches include the N patches.
  • a device for encoding point cloud data may include:
  • Memory for storing point cloud data.
  • An encoder configured to execute the first aspect or any possible design of the first aspect, or the point cloud encoding method described in the third aspect or any possible design of the third aspect.
  • an encoding device includes a memory and a processor.
  • the memory is used to store program code; the processor is used to call the program code to execute the first aspect or any possible design of the first aspect, or the third aspect or any possible design of the third aspect Point cloud coding method.
  • Memory for storing point cloud data.
  • a computer-readable storage medium stores program code, and when the program code is run on a computer, the computer is caused to execute the first aspect or any of the first aspect A possible design, or part or all of the steps of the point cloud coding method described in the third aspect or any one of the possible designs of the third aspect.
  • the term “at least one (seed / frame)” in the embodiments of the present application includes one (seed frame) or a plurality (seed frames).
  • “Multiple (seed frame)” means two (seed frame) or two (seed frame) or more.
  • at least one of A, B, and C includes: A alone, B alone, A and B simultaneously, A and C, B and C, and A, B, and C.
  • the term “and / or” in the embodiments of the present application is only an association relationship describing an associated object, and indicates that there can be three types of relationships. For example, A and / or B can mean: A exists alone, and A and B, there are three cases of B alone.
  • character “/” in the embodiments of the present application generally indicates that the related objects before and after are an “or” relationship. In the formula, the character "/" means division operation, such as A / B means A divided by B.
  • FIG. 1 is a schematic flowchart of a packaging method applicable to an example of an embodiment of the present application
  • FIG. 4 is a schematic block diagram of a point cloud decoding system applicable to an example of an embodiment of the present application
  • FIG. 7 is a schematic block diagram of a decoder that can be used in an example of an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a point cloud encoding method according to an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of another point cloud encoding method according to an embodiment of the present application.
  • FIG. 12 is a schematic block diagram of an encoder according to an embodiment of the present application.
  • FIG. 13 is a schematic block diagram of a decoder according to an embodiment of the present application.
  • FIG. 14 is a schematic block diagram of a decoding device according to an embodiment of the present application.
  • the encoder In a moving picture expert group (MPEG) point cloud encoding method, the encoder first divides the point cloud to be encoded (that is, the current frame or the current frame point cloud) into several patches according to certain criteria. These patches are mutually related There are no overlapping areas. Then, each patch is projected from a three-dimensional space to a two-dimensional plane to obtain a two-dimensional image (that is, the occupation map of the patch). Next, the occupancy maps of all patches (or the occupancy maps of the reduced resolution patches) are closely arranged on a two-dimensional image according to some rules to obtain the current frame occupancy map. This method of arranging the occupancy graphs of a patch is called packing.
  • MPEG moving picture expert group
  • the patches of the point cloud are arranged in a specific order, for example, in descending order of the width / height of the occupancy map of each patch; then, the occupancy map of the patch is inserted into the point in order according to the order of the arranged patches. From the available area of the cloud occupancy map, get the occupancy map of the point cloud. Subsequently, the current frame depth map and the current frame texture map are generated in the packing order.
  • the current frame depth map is a two-dimensional image generated by the projection depth of each patch in the packing order.
  • the current frame texture map is a two-dimensional image generated by the patched texture maps in the packing order.
  • the current frame occupancy map is a two-value two-dimensional image, which is used to indicate whether each pixel position of the two-dimensional image is occupied by a point in the point cloud.
  • the resolution of the current frame occupancy map is lower than the resolution of the current frame depth map and the current frame texture map. For the convenience of description, the following descriptions are taken without regard to whether or not the resolution is reduced.
  • FIG. 1 In order to improve the encoding or compression efficiency, a packaging method is shown in Figure 1.
  • the method shown in Figure 1 includes the following steps:
  • Step 1 Obtain the global matching patch of the point cloud group.
  • the point cloud group includes N-frame point clouds. N ⁇ 2, N is an integer.
  • the global matching patch of a frame point cloud refers to a patch having a matching relationship in a point cloud of the frame and a point cloud in each frame other than the frame point cloud in the point cloud group.
  • a patch that has a matching relationship with the patch is a global matching patch.
  • a patch having a matching relationship with a patch is a matching patch of a target patch, and the target patch is the patch or a matching patch of the patch.
  • the matching patch of one patch in a frame of point cloud in another frame of point cloud may be the largest intersection (IoU) in the point cloud of the other frame with the patch and the A patch with IoU greater than or equal to a preset threshold.
  • the union occupancy graph can also be called a union occupancy graph.
  • the N global matching patches included in a set are patches that have a matching relationship across the N-frame point clouds. It can be understood that each global matching patch in the N global matching patches comes from (or belongs to) a frame of point clouds, Different global matching patches are from different point clouds, and the N global matching patches have a matching relationship.
  • a union occupation map corresponding to a set to which a global matching patch belongs overwrites the occupation map of the global matching patch.
  • the method may further include:
  • Step 3 The M union set occupation maps are packaged to obtain a global occupation map (ie, the final occupation map).
  • the global occupancy map is used to determine the positions of M union occupancy maps in the global occupancy map.
  • the packaging process can be considered as a process of updating the global occupancy graph.
  • the M union set occupancy graphs are arranged in the available space of the global occupancy graph in order from top to bottom and from left to right to obtain the final Occupation diagram.
  • Step 4 Pack each point cloud in the N-frame point cloud to obtain an N-frame point cloud occupation map.
  • the position (first position) of the occupancy map of the m-th global matching patch in the n-th point cloud in the occupancy map of the n-th point cloud corresponds to the m-th union in the M-occupancy occupancy graph.
  • the position (second position) of the graph in the global occupancy graph 1 ⁇ n ⁇ N, where n is an integer.
  • the first position corresponding to the second position may include: the coordinate value of the first position is the same as the coordinate value of the second position; or the coordinate value of the first position in its coordinate system and the coordinate value of the second position are in its coordinates The system is substantially the same; or the coordinate value of the first position is different from the coordinate value of the second position, but the position range where the second position is located covers the position range where the first position is located.
  • the coordinate value of the first position can be characterized by the position coordinates of the mth globally matched patch occupation map in the nth frame point cloud occupation map in the nth frame point cloud's occupation map.
  • the coordinate value of the second position can be characterized by the position coordinates of the mth union set occupation map in the global occupation map.
  • the position range where the first position is located is the area occupied by the m-th global matching patch occupancy map
  • the position range where the second position is located is the area occupied by the m-th union occupancy map.
  • FIG. 2 and FIG. 3 are process schematic diagrams based on an example of the packaging method provided in FIG. 1.
  • the point cloud group includes 4 frames of point clouds
  • each frame of point clouds includes 10 patches
  • the global matching patches are: patch11, patch12, and patch13 in point cloud 1; patch21 in point cloud 2 , Patch22, and patch23; patch31, patch32, and patch33 in point cloud 3; patch41, patch42, and patch43 in point cloud 4.
  • patch11, patch21, patch31, and patch41 have a matching relationship
  • these patches form set 1.
  • patch12, patch22, patch32, and patch42 have a matching relationship
  • these patches form set 2.
  • patch13, patch23, patch33, and patch43 have a matching relationship
  • these patches form set 3. That is to say, the above-mentioned M sets are specifically sets 1 to 3, and each set includes 4 patches having a matching relationship.
  • the global occupancy graph obtained based on this example can be shown in FIG. 2.
  • the larger rectangle in FIG. 2 represents the global occupancy graph, and the ellipse, triangle, and smaller rectangle in FIG. 2 respectively represent the patch occupancy graph of the union set corresponding to sets 1 to 3.
  • the occupancy map of point cloud 1 may be as shown in (a) of FIG. 3, and the occupancy map of point cloud 2 may be as shown in (b) of FIG. 3.
  • FIG. 3 only illustrates the occupancy map of the point cloud 1 and the partial occupancy map of the point cloud 2, and does not illustrate the occupancy maps of the point cloud 3 and the point cloud 4.
  • the encoder needs to encode the information of each patch in a frame of point cloud into the code stream. In this way, a large code stream transmission overhead is caused. For this reason, the embodiments of the present application provide a point cloud encoding and decoding method and a codec, so as to save code stream transmission overhead without affecting the encoding and decoding performance.
  • the coordinate system defined in the point cloud decoding system is explained first.
  • the horizontal axis is the U0 axis and the vertical axis is the V0 axis.
  • the horizontal axis is generally defined as the U1 axis
  • the vertical axis is the V1 axis
  • the coordinate axis perpendicular to the U1 and V1 axes is the D1 axis. That is, the U1 axis is the X axis in the world coordinate system, the V1 axis is the Y axis in the world coordinate system, and the D axis is the Z axis in the world coordinate system.
  • the occupancy map of the frame level may be the occupancy map of the point cloud.
  • the occupancy map at the patch level is a union occupancy map
  • the occupancy map at the frame level may be a global occupancy map.
  • FIG. 4 is a schematic block diagram of a point cloud decoding system 1 that can be used in an example of an embodiment of the present application.
  • the terms "point cloud coding” or “coding” may generally refer to point cloud coding or point cloud decoding.
  • the encoder 100 of the point cloud decoding system 1 may encode the point cloud to be encoded according to any one of the point cloud encoding methods proposed in this application.
  • the decoder 200 of the point cloud decoding system 1 may decode the point cloud to be decoded according to the point cloud decoding method corresponding to the point cloud encoding method used by the encoder.
  • the point cloud decoding system 1 includes a source device 10 and a destination device 20.
  • the source device 10 generates encoded point cloud data. Therefore, the source device 10 may be referred to as a point cloud encoding device.
  • the destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may be referred to as a point cloud decoding device.
  • Various implementations of the source device 10, the destination device 20, or both may include one or more processors and a memory coupled to the one or more processors.
  • the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), read-only memory (EEPROM) ), Flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures accessible by a computer, as described herein.
  • RAM random access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • EEPROM read-only memory
  • Flash memory or any other medium that can be used to store the desired program code in the form of instructions or data structures accessible by a computer, as described herein.
  • the source device 10 and the destination device 20 may include various devices including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets, such as so-called “smart” phones, etc. Cameras, televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.
  • the destination device 20 may receive the encoded point cloud data from the source device 10 via the link 30.
  • the link 30 may include one or more media or devices capable of moving the encoded point cloud data from the source device 10 to the destination device 20.
  • the link 30 may include one or more communication media that enable the source device 10 to send the encoded point cloud data directly to the destination device 20 in real time.
  • the source device 10 may modulate the encoded point cloud data according to a communication standard (e.g., a wireless communication protocol), and may transmit the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (eg, the Internet).
  • the one or more communication media may include a router, a switch, a base station, or other devices that facilitate communication from the source device 10 to the destination device 20.
  • the encoded data may be output from the output interface 140 to the storage device 40.
  • the encoded point cloud data can be accessed from the storage device 40 through the input interface 240.
  • the storage device 40 may include any of a variety of distributed or locally-accessed data storage media, such as a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), or a compact disc (read-only). only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded point cloud data.
  • the transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of the two.
  • the point cloud decoding system 1 illustrated in FIG. 4 is merely an example, and the technology of the present application is applicable to point cloud decoding (for example, point (Cloud encoding or point cloud decoding) device.
  • point cloud decoding for example, point (Cloud encoding or point cloud decoding) device.
  • data is retrieved from local storage, streamed over a network, and so on.
  • the point cloud encoding device may encode the data and store the data to a memory, and / or the point cloud decoding device may retrieve the data from the memory and decode the data.
  • encoding and decoding are performed by devices that do not communicate with each other, but only encode data to and / or retrieve data from memory and decode data.
  • the source device 10 includes a data source 120, an encoder 100, and an output interface 140.
  • the output interface 140 may include a modulator / demodulator (modem) and / or a transmitter (or a transmitter).
  • the data source 120 may include a point cloud capture device (e.g., a camera), a point cloud archive containing previously captured point cloud data, a point cloud feed interface to receive point cloud data from a point cloud content provider, and / or Computer graphics systems for generating point cloud data, or a combination of these sources of point cloud data.
  • the encoder 100 may encode point cloud data from the data source 120.
  • the source device 10 sends the encoded point cloud data directly to the destination device 20 via the output interface 140.
  • the encoded point cloud data may also be stored on the storage device 40 for later access by the destination device 20 for decoding and / or playback.
  • the destination device 20 includes an input interface 240, a decoder 200, and a display device 220.
  • the input interface 240 includes a receiver and / or a modem.
  • the input interface 240 may receive the encoded point cloud data via the link 30 and / or from the storage device 40.
  • the display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data.
  • the display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer (MUX-DEMUX) unit or other hardware and software to handle encoding of both audio and video in a common or separate data stream.
  • MUX-DEMUX multiplexer-demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).
  • the encoder 100 and the decoder 200 may each be implemented as any of a variety of circuits such as one or more microprocessors, digital signal processors (DSPs), and application specific integrated circuits (applications) specific integrated circuit (ASIC), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the present application is implemented partially in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium and may use one or more processors to execute the instructions in hardware Thus implementing the technology of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, any of which may be integrated as a combined encoder / decoder in a corresponding device (Codec).
  • codec codec
  • This application may generally refer to the encoder 100 as “signaling” or “sending” certain information to another device, such as the decoder 200.
  • the terms “signaling” or “sending” may generally refer to the transmission of syntax elements and / or other data to decode the compressed point cloud data. This transfer can occur in real time or almost real time. Alternatively, this communication may occur after a period of time, such as may occur when a syntax element is stored in a coded bit stream to a computer-readable storage medium at the time of encoding, and the decoding device may then after the syntax element is stored to this medium Retrieve the syntax element at any time.
  • FIG. 5 it is a schematic block diagram of an encoder 100 that can be used in an example of an embodiment of the present application.
  • FIG. 5 is described by using an MPEG (Moving Picture Expert Group) point cloud compression (PCC) coding framework as an example.
  • the encoder 100 may include a patch information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a first filling module 105, an image or video-based encoding module 106, and an occupation The graph coding module 107, the auxiliary information coding module 108, the multiplexing module 109, and the like.
  • the encoder 100 may further include a point cloud filtering module 110, a second filling module 111, a point cloud reconstruction module 112, and the like. among them:
  • the patch information generating module 101 is configured to divide a point cloud of a frame into multiple patches by using a certain method, and obtain related information of the generated patches.
  • patch refers to a set of partial points in a frame of point cloud.
  • one connected area corresponds to one patch.
  • the relevant information of the patch may include but is not limited to at least one of the following information: the number of patches divided by the point cloud, the position information of the patch in the three-dimensional space, the index of the normal axis of the patch, and the projection of the patch from the three-dimensional space Depth map generated in two-dimensional space, depth map size of the patch (such as width and height of the depth map), occupancy map generated by the projection of the patch from three-dimensional space to two-dimensional space, etc.
  • Part of the relevant information such as the number of patches divided by the point cloud, the index of the normal axis of the patch, the depth map size of the patch, the position information of the patch in the point cloud, the size information of the patch occupancy map, etc. , Can be sent to the auxiliary information encoding module 108 as auxiliary information for encoding (that is, compression encoding).
  • the occupancy map of each patch can be sent to the packaging module 102 for packaging.
  • the specific position information of each patch in the point cloud occupancy map and the depth map of each patch can be sent to the depth map generation module 103.
  • the occupancy map of the point cloud may be filled by the second filling module 111 and sent to the occupancy map encoding module 107 for encoding.
  • the occupancy map of the point cloud can be used to guide the depth map generation module 103 to generate the depth map of the point cloud and the guided texture map generation module 104 to generate the texture map of the point cloud.
  • the packaging module 102 may also send auxiliary information such as the union occupation map generated during the packaging process to the auxiliary information encoding module 108 to transmit the union occupation map.
  • auxiliary information is programmed into the code stream. In this way, it is not necessary to transmit the relevant information of the global matching patch corresponding to the union occupancy map through the code stream, so the code stream overhead can be saved.
  • the module for coding the auxiliary information of the union occupation map into the code stream may also be a module independent of the auxiliary information coding module 108, which is not limited in the embodiment of the present application.
  • a depth map generating module 103 is configured to generate a depth map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud, and depth information, and send the generated depth map to the first filling module 105 To fill the blank pixels in the depth map to get the filled depth map.
  • the texture map generating module 104 is configured to generate a texture map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud, and texture information, and send the generated texture map to the first filling module 105 To fill the blank pixels in the texture map to get the filled texture map.
  • the filled depth map and the filled texture map are sent by the first filling module 105 to the image or video-based encoding module 106 to perform image or video-based encoding.
  • the image or video-based encoding module 106, the occupancy map encoding module 107, and the auxiliary information encoding module 108 send the obtained encoding result (that is, the code stream) to the multiplexing module 109 to merge into a code stream.
  • the code stream may be sent to the output interface 140.
  • the encoding result (ie code stream) obtained by the image or video based encoding module 106 is sent to the point cloud reconstruction module 112 for point cloud reconstruction to obtain a reconstructed point cloud (that is, a reconstructed Point cloud geometry information).
  • the geometric information of the point cloud refers to the coordinate values of points in the point cloud (for example, each point in the point cloud) in a three-dimensional space.
  • the point cloud reconstruction module 112 may also send the texture information of the point cloud and the reconstructed point cloud geometric information to the coloring module, and the coloring module is used to color the reconstructed point cloud to obtain the reconstructed point cloud. Texture information.
  • the texture map generating module 104 may further generate a texture map of the point cloud based on the information obtained by filtering the reconstructed point cloud geometric information through the point cloud filtering module 110.
  • FIG. 6 it is a schematic diagram of a point cloud, a patch, and an occupation map of a point cloud applicable to an example of the embodiment of the present application.
  • (a) in FIG. 6 is a schematic diagram of a point cloud
  • (b) in FIG. 6 is a schematic diagram of a patch based on the point cloud obtained in (a) in FIG. 6,
  • (c) in FIG. 6 is A schematic diagram of the occupancy map of the point cloud obtained by packaging the occupancy map of each patch obtained by mapping each patch shown in (b) of FIG. 6 to a two-dimensional plane.
  • the occupancy map of the point cloud shown in (c) in FIG. 6 may be obtained according to the method shown in FIG. 1 above.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupation map decoding module 203, an auxiliary information decoding module 204, a point cloud reconstruction module 205, and a point cloud filtering module. 206 and the texture information reconstruction module 207 of the point cloud. among them:
  • the image or video-based decoding module 202 is configured to decode the received encoded texture map and the encoded depth map; and then send the decoded texture map information to the point cloud texture information reconstruction module 207, Send the decoded depth map information to the point cloud reconstruction module 205.
  • the occupancy map decoding module 203 is configured to decode the received code stream containing the encoded occupancy map, and send the decoded occupancy map information to the point cloud reconstruction module 205.
  • the auxiliary information decoding module 204 is configured to decode the received encoded auxiliary information, and send the decoded information indicating the auxiliary information to the point cloud reconstruction module 205.
  • the point cloud texture information reconstruction module 207 is configured to reconstruct the point cloud texture information to obtain a reconstructed point cloud.
  • the decoder 200 shown in FIG. 7 is only an example. In specific implementation, the decoder 200 may include more or fewer modules than those shown in FIG. 7. This embodiment of the present application does not limit this.
  • FIG. 8 it is a schematic flowchart of a point cloud encoding method according to an embodiment of the present application.
  • the execution body of the method shown in FIG. 8 may be the encoder 100.
  • the method shown in FIG. 8 includes the following steps:
  • the point cloud group includes at least two frame point clouds, for example, at least two frame point clouds that are continuous in time.
  • the point cloud group may be one or more GOFs, which generally refers to one GOF.
  • the frame number of the point cloud in GOF is configurable. This embodiment of the present application does not limit the number of frames of a point cloud included in a GOF.
  • a GOF may include a 32-frame point cloud.
  • the determination method of GOF refer to the prior art.
  • the point cloud group may be a sub-GOF, and the sub-GOF may be composed of time-continuous multi-frame point clouds in one GOF.
  • a GOF may include at least two sub-GOFs, and the number of frames of the point cloud contained in any two of the at least two sub-GOFs may be equal or unequal.
  • the encoder and the decoder may divide one GOF into at least two sub-GOFs based on the same predefined (eg, predefined through a protocol) manner.
  • the encoder may divide a GOF into at least two sub-GOFs (for example, divide a GOF into at least two sub-GOFs based on the continuity between the content of point clouds contained in the GOF), and then represent the at least two sub-GOFs.
  • the position information in the GOF that is, the information of which point clouds each sub-GOF includes
  • the decoder obtains the positions of the at least two sub-GOFs in the GOF by analyzing the information obtained from the code stream.
  • the information indicating the positions of the at least two sub-GOFs in the GOF may include: a number of a start frame (that is, a point cloud of the first frame) of each sub-GOF and / or a number of frames included in each sub-GOF, etc . It can be understood that the number of the start frame of the first sub-GOF may not be coded into the code stream and / or the number of frames contained in one GOF may not be coded into the code stream. This optional embodiment may be applied to a scenario in which the number of frames included in each of the at least two sub-GOFs is the same, and it may also be applied to the number of frames included in some or all of the at least two sub-GOFs. Scene.
  • the above syntax element may further include information indicating a position of each point cloud in the sub-GOF in the GOF to which the sub-GOF belongs, For example, the number of the start frame contained in the sub-GOF and / or the number of frames of the point cloud contained in the sub-GOF.
  • the information indicating the position of the at least two sub-GOFs in the GOF may include: the number of frames of the point cloud included in one sub-GOF , Or the number of child GOFs contained in a GOF.
  • the number of union set occupation maps of the point cloud group may be one or more.
  • the number of union set occupation maps of the point cloud group is M.
  • the union occupancy map in S101 may be any union occupancy map in the point cloud group.
  • Each of the N patches in S101 can be considered as a global matching patch described above.
  • Each of the N patches is from a point cloud, and different patches are from different point clouds.
  • global matching patch please refer to the above, which will not be repeated here.
  • the auxiliary information of the union occupancy graph refers to information used to describe the union occupancy graph, such as but not limited to including at least one of the following information: the coordinate value of the union occupancy graph in the global occupancy graph, the union The size information of the set occupancy map in the global occupancy map and the coordinate values of the union occupancy map corresponding to the three-dimensional space.
  • the global occupancy map is an occupancy map obtained by packing multiple union occupancy maps in a point cloud group. For the related description of the global occupancy graph, please refer to the above, which will not be repeated here.
  • the coordinate values of the union occupation map in the global occupation map include: the coordinate values of the union occupation map on the U0 axis of the global occupation map (such as u0 of the union occupation map), and / or the V0 axis
  • the coordinate value (such as v0 of the union occupation graph).
  • the encoder may use the minimum or maximum value of u0 of the N occupation patches corresponding to the union occupation map as u0 of the union occupation map; and / or the N occupations corresponding to the union occupation map.
  • the minimum value or maximum value of v0 of the patch occupation graph is used as v0 of the union occupation graph.
  • the size information of the union occupation map in the global occupation map includes: the size information of the union occupation map on the U0 axis of the global occupation map (such as the size U0 of the union occupation map), and / or the V0 axis Size information (such as sizeV0 of the union occupation map).
  • the encoder may use the maximum value of the sizeU0 of the occupation map corresponding to the union occupation map as the sizeU0 of the union occupation map, and / or the occupation of the N patches corresponding to the union occupation map.
  • the maximum value of sizeU0 of the graph is taken as the sizeV0 of the union occupation graph.
  • the coordinate value corresponding to the union occupation map in the three-dimensional space refers to the minimum (or maximum, etc.) coordinate value of the N patches corresponding to the union occupation map in the three-dimensional space.
  • the coordinate value corresponding to the three-dimensional space of the union occupation map may include: the minimum coordinate value (such as the minimum u1 value) of the N patches in the three-dimensional U1 axis, and the V1 axis.
  • the auxiliary information of the union occupation map is used to determine the auxiliary information of the N patches, which can be understood as: the auxiliary information of the union occupation map is used by the decoder to determine the auxiliary information of the N patches.
  • the N patches in S101 are referred to as the N patches corresponding to the union occupation map in S101. It is unified here and will not be described in detail below.
  • the above S101 to S102 describe the point cloud encoding method from the perspective of N patches corresponding to a union occupancy map.
  • any one or more union occupancy maps in a point cloud group that is, the global
  • Each of the matching (patch) methods can perform a point cloud coding method based on S101 to S102.
  • the embodiments of the present application do not limit the point cloud encoding method to which they are applicable.
  • any point cloud encoding method provided in the prior art, such as an MPEG point cloud encoding method may be used for encoding.
  • the other patches may include all non-global matching patches, and may further include other global matching patches in addition to the global matching patches that perform the point cloud coding method according to the above S101-S102.
  • the encoder may execute S101-S102 for N patches corresponding to one union-occupied graph, and then execute S101-102 for N patches corresponding to another union-occupied graph, until After executing all the patches corresponding to the union occupancy graph that needs to be encoded using the technical solution provided by the embodiment of the present application.
  • the encoder may sequentially or simultaneously respond to the patch corresponding to the union occupancy map that needs to be encoded using the technical solution provided by the embodiment of the present application.
  • the encoder After executing the above S101, the encoder may sequentially or simultaneously respond to the union occupancy map.
  • the patch executes the above S102, which is not limited in the embodiment of the present application.
  • the position information of the union occupancy graph in the global occupancy graph corresponds to the above-mentioned N patches.
  • the position information of the occupancy map in the occupancy map of the point cloud to which it belongs, and / or the union occupancy map is the union of the occupancy maps of the N patches, etc. Therefore, it can be represented or approximated by the auxiliary information of the union occupancy map. Represents auxiliary information for the N patches.
  • the encoder encodes auxiliary information of the union occupation map into a code stream, so that the decoder determines N corresponding to the union occupation map based on the auxiliary information of the union occupation map.
  • Auxiliary information for patch In this way, compared with the technical solution of directly transmitting the auxiliary information of the N patches in the code stream in the prior art, the technical solution provided in the embodiment of the present application helps to save code stream transmission without affecting the codec performance. Overhead.
  • the method shown in FIG. 8 further includes the following step S103:
  • S103 Reconstruct the N-frame point cloud according to the auxiliary information of multiple patches (such as all patches) in the N-frame point cloud, where the multiple patches include the N patches.
  • the syntax element in the embodiment shown in FIG. 8 further includes a difference between corresponding auxiliary information of one or more patches (for example, all patches) in the N patches and auxiliary information of the union occupation map.
  • the corresponding auxiliary information of the one or more patches refers to auxiliary information of the same type as the auxiliary information of the union occupation map.
  • the auxiliary information of the union occupancy graph and the corresponding auxiliary information of the one or more patches are all u0 or v0; or, all sizeU0 or all sizeV0; or all u1 or all d1 or both v1.
  • This optional implementation can be understood as transmitting auxiliary information of one or more patches corresponding to the union occupation map in a differential manner.
  • the auxiliary information of the union occupancy graph and the auxiliary information of the N patches may not be exactly the same. Therefore, transmitting the auxiliary information of one or more patches corresponding to the occupancy map in a differential manner helps to improve the accuracy and / or setting flexibility of the auxiliary information of the one or more patches in the N patches. For example, the accuracy of u0, v0, u1, v1, and / or d1 of the one or more patches may be improved.
  • the setting flexibility of the sizeU0 and / or sizeV0 of the one or more patches may be improved, that is, it is not necessary to limit the sizeU0 of the one or more patches to the sizeU0 of the union occupation map, and / or it is not necessary
  • the sizeV0 limited to the one or more patches is equal to the sizeV0 of the union occupation graph.
  • whether the syntax element includes the difference between the corresponding auxiliary information of one or more of the above N patches (for example, all patches) and the auxiliary information of the union occupancy graph can be predefined, such as Predefined through the protocol, or the encoder may encode the indication information indicating whether the difference is included into the code stream and send it to the decoder.
  • FIG. 9 it is a schematic flowchart of a point cloud encoding method according to an embodiment of the present application.
  • the execution body of the method shown in FIG. 9 may be the decoder 200.
  • the method shown in FIG. 9 includes the following steps:
  • S201 Parse the bitstream to obtain the syntax element.
  • the syntax element includes auxiliary information of the union set occupation map of the point cloud group, the point cloud group includes N frame point clouds, N ⁇ 2, N is an integer, and the union occupation map Is the union of the occupancy maps of N patches with matching relationships in the N-frame point cloud.
  • the syntax element further includes a difference between corresponding auxiliary information of one or more patches (for example, all patches) in the N patches and auxiliary information of the union occupation map.
  • This optional manner corresponds to the embodiment in which the syntax element in the embodiment shown in FIG. 8 includes the difference.
  • S202 Determine the auxiliary information of the N patches according to the auxiliary information of the union occupation map obtained through analysis.
  • the decoder may use the auxiliary information of the union set occupation map as the corresponding auxiliary information of each of the N patches.
  • the syntax element includes auxiliary information of the union occupancy graph, and includes corresponding auxiliary information of one or more patches in the corresponding N patches of the union occupancy graph
  • the difference between the auxiliary information of the union occupation map and the auxiliary information of the patch may be based on the auxiliary information of the union occupation map and the corresponding auxiliary information of the patch for each patch in the one or more patches.
  • the auxiliary information corresponding to the patch is determined. Specifically, the auxiliary information of the union set occupation graph and the difference are summed to obtain the corresponding auxiliary information of the patch.
  • the decoder may use the auxiliary information of the union occupation map as Corresponding auxiliary information of the N patches.
  • the decoder can obtain the auxiliary information of the patch by analyzing the code stream.
  • the method can refer to the prior art.
  • S203 Reconstruct the N-frame point cloud based on auxiliary information of multiple patches (for example, all patches) in the N-frame point cloud; wherein the plurality of patches include the N patches.
  • auxiliary information of multiple patches for example, all patches
  • the plurality of patches include the N patches.
  • any point cloud decoding method provided in this embodiment corresponds to the corresponding point cloud encoding method in the embodiment shown in FIG. 8. Therefore, for descriptions of related terms, specific implementations, and beneficial effects in this embodiment, reference may be made to the foregoing.
  • the embodiment of the point cloud coding method is not repeated here.
  • the point cloud group includes GOF or sub-GOF.
  • the decoder obtains the position of the sub-GOF in the GOF to which the sub-GOF belongs, reference may be made to the foregoing, and details are not described herein again.
  • FIG. 10 it is a schematic flowchart of a point cloud encoding method according to an embodiment of the present application.
  • the execution body of the method shown in FIG. 10 may be the encoder 100.
  • the method shown in FIG. 10 includes the following steps:
  • the target point cloud encoding method may be any one of the point cloud encoding methods provided in the embodiments of the present application, such as the point cloud encoding method shown in FIG. 8.
  • the point cloud group includes N frame point clouds, N ⁇ 2, and N is an integer; the N patches are N patches having a matching relationship in the N frame point clouds.
  • the encoder may support at least two encoding methods, and one of the at least two may be any of the point cloud encoding methods provided in the embodiments of the present application, as shown in the point cloud encoding method shown in FIG. 8, Other other point cloud encoding methods may be any existing point cloud encoding method provided in the future.
  • the method further includes the following step S303:
  • S303 Code the auxiliary information of the N patches according to the target point cloud coding method.
  • the auxiliary information of the N patches is encoded according to the point cloud encoding method shown in FIG. 8.
  • This embodiment provides a technical solution for selecting a target encoding method, and the technical solution can be applied to a scenario in which an encoder supports at least two point cloud encoding methods. That is, the embodiment of the present application supports a technical solution in which an encoder selects a target point cloud encoding method from at least two candidate point cloud encoding methods.
  • FIG. 11 it is a schematic flowchart of a point cloud decoding method according to an embodiment of the present application.
  • the execution body of the method shown in FIG. 11 may be the decoder 200.
  • the method shown in FIG. 11 includes the following steps:
  • the target point cloud decoding method may be any one of the point cloud decoding methods provided in the embodiments of the present application, such as the point cloud decoding method shown in FIG. 9.
  • the point cloud group includes N frame point clouds, N ⁇ 2, and N is an integer; the N patches are N patches having a matching relationship in the N frame point clouds.
  • the target point cloud decoding method is used to process the auxiliary information of the N patches. For details, refer to the foregoing S201 to S203.
  • group_of_frames_size in Table 2 indicates the number of frames of the point cloud included in the GOF
  • subGOFsize indicates the number of frames of the point cloud included in the sub-GOF.
  • the encoder and the decoder may be divided into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules may be implemented in the form of hardware or software functional modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be another division manner.
  • FIG. 12 is a schematic block diagram of an encoder 120 according to an embodiment of the present application.
  • the encoder 120 may be used to perform the point cloud encoding method shown in FIG. 8 or FIG. 10.
  • the encoder 120 includes a packing module 1201 and an auxiliary information encoding module 1202.
  • the packing module 1201 is configured to obtain auxiliary information of a union set occupation map of a point cloud group; the point cloud group includes N frame point clouds, N ⁇ 2, N is an integer, and the union occupation map is the N The union of the occupancy maps of N patches with a matching relationship in the frame point cloud; the auxiliary information of the occupancy map of the union set is used to determine the auxiliary information of the N patches.
  • the auxiliary information encoding module 1202 is configured to encode a syntax element into a code stream, where the syntax element includes auxiliary information of the union occupation map.
  • the packaging module 1201 may be used to execute S101, and the auxiliary information encoding module 1202 may be used to execute S102.
  • the auxiliary information of the union occupation map includes at least one of the following information: the coordinate values of the union occupation map in the global occupation map, the size information of the union occupation map in the global occupation map, and the union occupation map Corresponds to the coordinate value in three-dimensional space.
  • the global occupancy map is an occupancy map obtained by packing multiple union occupancy maps in a point cloud group.
  • the packaging module 1201 is specifically configured to: occupy the occupancy map of the N patches at the corresponding point cloud.
  • the maximum or minimum coordinate value in the graph is used as the coordinate value of the union occupation graph in the global occupation graph;
  • the packing module 1201 is specifically configured to: the maximum coordinate value of the N patches in the corresponding point cloud Or the minimum coordinate value is used as the coordinate value corresponding to the three-dimensional space of the union occupation map.
  • the encoder 120 further includes a point cloud reconstruction module 1203, configured to reconstruct the N frame point cloud according to auxiliary information of multiple patches in the N frame point cloud; wherein the plurality of patches include the N A patch.
  • the point cloud reconstruction module 1203 may be used to execute S103.
  • the point cloud group is a GOF or a sub-GOF
  • the sub-GOF includes a continuous multi-frame point cloud in one GOF.
  • the above syntax element further includes: information indicating a position of a point cloud (such as a point cloud per frame) included in the sub GOF in a GOF to which the sub GOF belongs.
  • a position of a point cloud such as a point cloud per frame
  • the number of the start frame contained in the sub-GOF, and / or the number of frames of the point cloud contained in the sub-GOF For example, the number of the start frame contained in the sub-GOF, and / or the number of frames of the point cloud contained in the sub-GOF.
  • the packaging module 1201 is configured to determine instruction information that indicates whether to use target point cloud encoding method to encode auxiliary information of N patches in the point cloud group.
  • the target point cloud encoding method includes the above-mentioned figure.
  • the auxiliary information encoding module 1202 is configured to encode the instruction information into a code stream.
  • the encoder 120 further includes a point cloud reconstruction module 1203, configured to reconstruct the N frame point cloud according to auxiliary information of multiple patches in the N frame point cloud; the multiple patches include the N A patch.
  • the packaging module 1201 may be used to execute S301
  • the auxiliary information encoding module 1202 may be used to execute S302
  • the point cloud reconstruction module 1203 may be used to execute S303.
  • the packaging module 1201 may correspond to the packaging module 102 in FIG. 5, and the auxiliary information encoding module 1202 may correspond to the auxiliary information encoding module 108 in FIG. 5.
  • the point cloud reconstruction module 1203 may specifically correspond to the point cloud reconstruction module 112 in FIG. 5. It should be noted that the packaging module 1201 and the reconstruction module 1203 may be directly connected or indirectly connected through other functional modules. For an example of an indirect connection, refer to the packaging module 102 and the point cloud reconstruction module 112 in FIG. 5 Indirect connection relationship.
  • FIG. 13 is a schematic block diagram of a decoder 130 according to an embodiment of the present application.
  • the decoder 130 may be configured to perform the point cloud decoding method shown in FIG. 9 or FIG. 11.
  • the decoder 130 includes an auxiliary information decoding module 1301 and a point cloud reconstruction module 1302.
  • the auxiliary information decoding module 1301 is configured to parse a code stream to obtain a syntax element, the syntax element includes auxiliary information of a union occupation map of a point cloud group, the point cloud group includes N frame point clouds, and N ⁇ 2, N is an integer, and the union occupancy map is the union of the occupancy maps of N patches with matching relationships in the N-frame point cloud; the N patches are determined according to the auxiliary information of the union occupancy map obtained through analysis Auxiliary information.
  • the point cloud reconstruction module 1302 is configured to reconstruct the N frame point cloud based on auxiliary information of multiple patches in the N frame point cloud, where the plurality of patches include the N patches. For example, in conjunction with FIG. 9, the auxiliary information decoding module 1301 may be used to execute S201 and S202, and the point cloud reconstruction module 1302 may be used to execute S203.
  • the auxiliary information of the union occupation map includes at least one of the following information: the coordinate values of the union occupation map in the global occupation map, the size information of the union occupation map in the global occupation map, and the union occupation map Corresponds to the coordinate value in three-dimensional space.
  • the global occupancy map is an occupancy map obtained by packing multiple union occupancy maps in the point cloud group.
  • the point cloud group is a GOF or a sub-GOF
  • the sub-GOF includes a continuous multi-frame point cloud in one GOF.
  • the above syntax element further includes: information indicating a position of a point cloud (such as a point cloud per frame) contained in the sub-GOF in a GOF to which the sub-GOF belongs, for example, The number of the start frame contained in the sub-GOF, and / or the number of frames of the point cloud contained in the sub-GOF.
  • the auxiliary information decoding module 1301 is further configured to determine, based on the parsed information indicating the position of the point cloud included in the sub-GOF in the GOF to which the sub-GOF belongs, the point cloud included in the sub-GOF (such as each frame) Point cloud) in the GOF to which the child GOF belongs.
  • the auxiliary information decoding module 1301 is configured to parse the code stream to obtain instruction information, which indicates whether to use the target point cloud decoding method to decode the auxiliary information of the N patches in the point cloud group.
  • the point cloud decoding method includes the point cloud decoding method shown in FIG. 11 described above; the point cloud group includes N frame point clouds, N ⁇ 2, and N is an integer; the N patches are N having a matching relationship in the N frame point clouds. A patch.
  • the instruction information indicates that the target point cloud decoding method is used for decoding
  • the auxiliary information decoding module 1301 and the point cloud reconstruction module 1302 use the target point cloud decoding method to process the auxiliary information of the N patches. For example, in conjunction with FIG.
  • the auxiliary information decoding module 1301 may be used to execute S401, and the auxiliary information decoding module 1301 and the point cloud reconstruction module 1302 may be used to execute S402.
  • the auxiliary information decoding module 1301 and the point cloud reconstruction module 1302 may be used to execute S203.
  • the auxiliary information decoding module 1301 may correspond to the auxiliary information decoding module 204 in FIG. 7, and the point cloud reconstruction module 1302 may correspond to the point cloud reconstruction module 205 in FIG. 7.
  • each module in the codec according to the embodiment of the present application is a functional body that implements various execution steps included in the point cloud encoding and decoding method corresponding to the present application, that is, it has the function of realizing the point cloud corresponding to the present application.
  • Each step in the encoding and decoding method, and the main body of the expansion and deformation of these steps please refer to the description of the image filtering method in this article for details. For the sake of brevity, this article will not repeat them.
  • FIG. 14 is a schematic block diagram of an implementation manner of an encoding device or a decoding device (referred to as a decoding device 160 or a point cloud decoding device 160) used in the embodiment of the present application.
  • the decoding device 160 may include a processor 1610, a memory 1630, and a bus system 1650.
  • the processor 1610 and the memory 1630 are connected through a bus system 1650.
  • the memory 1630 is configured to store instructions.
  • the processor 1610 is configured to execute the instructions stored in the memory 1630 to perform various point cloud encoding or decoding methods described in this application. , Especially a method for filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.
  • the processor 1610 may be a central processing unit (CPU), and the processor 1610 may also be another general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate, or transistor. Logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 1630 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as the memory 1630.
  • the memory 1630 may include code and data 1631 accessed by the processor 1610 using the bus system 1650.
  • the memory 1630 may further include an operating system 1633 and an application program 1635, which includes a processor 1610 that allows the processor 1610 to perform the point cloud encoding or decoding method described in this application (especially the current image based on the block size of the current image Block filtering method) at least one program.
  • the application program 1635 may include applications 1 to N, which further includes a point cloud encoding or decoding application (referred to as a point cloud decoding application) that performs the point cloud encoding or decoding method described in this application.
  • the bus system 1650 may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are marked as the bus system 1650 in the figure.
  • the decoding device 160 may further include one or more output devices, such as a display 1670.
  • the display 1670 may be a tactile display that incorporates the display with a tactile unit operatively sensing a touch input.
  • the display 1670 may be connected to the processor 1610 via a bus system 1650.
  • Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol) .
  • computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave.
  • a data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used to implement the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or may be used to store instructions or data structures Any form of desired program code and any other medium accessible by a computer.
  • any connection is properly termed a computer-readable medium.
  • a coaxial cable is used to transmit instructions from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are instead directed to non-transitory tangible storage media.
  • magnetic and optical discs include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs, where magnetic discs typically reproduce data magnetically and optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • the term "processor” as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein.
  • the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or Into the combined codec.
  • the techniques can be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logic elements.
  • the techniques of this application may be implemented in a wide variety of devices or devices, including a wireless handset, an integrated circuit (IC), or a group of ICs (eg, a chipset).
  • IC integrated circuit
  • Various components, modules, or units are described in this application to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units.
  • the various units may be combined in a codec hardware unit in combination with suitable software and / or firmware, or through interoperable hardware units (including one or more processors as described above) provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请公开了点云编解码方法和编解码器，涉及编解码技术领域，用于提供有效地对点云数据进行编解码的技术方案。该点云编码方法包括：获取点云组的并集占用图的辅助信息；其中，该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；该并集占用图的辅助信息用于确定该N个patch的辅助信息；将语法元素编入码流，该语法元素包括该并集占用图的辅助信息。本申请提供的技术方案有助于在不影响编解码性能的前提下，节省码流传输开销。

Description

点云编解码方法和编解码器
本申请要求于2018年09月30日提交国家知识产权局、申请号为201811166410.8、申请名称为“点云编解码方法和编解码器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及编解码技术领域，尤其涉及点云(point cloud)编解码方法和编解码器。
背景技术
随着3d传感器(例如3d扫描仪)技术的不断发展，采集点云数据越来越便捷，所采集的点云数据的规模也越来越大，因此，如何有效地对点云数据进行编码，成为迫切需要解决的问题。
发明内容
本申请实施例提供了点云编解码方法和编解码器，用于提供有效地对点云数据进行编解码的技术方案，具体的，有助于在不影响编解码性能的前提下，节省码流传输开销。
第一方面，提供一种点云编码方法，包括：获取点云组的并集占用图的辅助信息；其中，该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个点云块(patch)的占用图的并集；该并集占用图的辅助信息用于确定(具体的，用于解码器确定)该N个patch的辅助信息(auxiliary information)；将语法元素编入码流，该语法元素包括该并集占用图的辅助信息。该技术方案的执行主体可以是编码器。
其中，该N个patch中的每个patch来自该N帧点云中的一帧点云，且不同patch来自不同的点云。例如，假设N＝4，则4帧点云中具有匹配关系的4个patch分别来自于或分别被包括于该4帧点云。与一个patch具有匹配关系的patch是目标patch的匹配patch，目标patch是该patch或者该patch的匹配patch。该N个patch中的每个patch均可以被称为是全局匹配patch(global matched patch)。
由于并集占用图与该并集占用图对应的N个patch的占用图之间的相关性，例如并集占用图在全局占用图中的位置信息，对应于上述N个patch的占用图在其所属的点云的占用图中的位置信息，和/或并集占用图是该N个patch的占用图的并集等，因此，可以通过并集占用图的辅助信息表示或近似表示该N个patch的辅助信息。这样，在不影响编解码性能的前提下，可以节省码流传输开销。
在一种可能的设计中，并集占用图的辅助信息包括以下信息中的至少一种：并集占用图在全局占用图中的坐标值、并集占用图在全局占用图中的尺寸信息和并集占用图对应在三维空间中的坐标值。全局占用图是对该点云组中的多个并集占用图进行打包得到的占用图。
其中，并集占用图在全局占用图中的坐标值包括：并集占用图的u0和/或v0。
其中，并集占用图在全局占用图中的尺寸信息包括：并集占用图的sizeU0和/或sizeV0。
其中，并集占用图对应在三维空间中的坐标值，是指该并集占用图对应的N个patch在三维空间中的最小(或最大)坐标值。以最小坐标值为例，并集占用图对应在三维空间中的坐标值可以包括：该N个patch在三维空间中的最小u1值、最小v1值和/或最小d1值。
在一种可能的设计中，上述N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值。该一个或多个patch的对应辅助信息是指与该并集占用图的辅助信息是同类型的辅助信息，例如均是u0等。该可能的设计使用差分方式传输并集占用图对应的一个或多个patch的辅助信息，这样，有助于提高N个patch中的该一个或多个patch的辅助信息的准确度和/或设置灵活性。具体实现的过程中，可选的，如果该N个patch中的某个patch的辅助信息与该并集占用图的辅助信息相同，则该某个patch的辅助信息与该并集占用图的辅助信息的差值为0，则该差值可以不被编入码流。
在一种可能的设计中，获取该点云组的并集占用图的辅助信息，包括：基于上述N个patch的辅助信息，获取该并集占用图的辅助信息。
在一种可能的设计中，基于该N个patch的辅助信息，获取该并集占用图的辅助信息，包括：
当该并集占用图的辅助信息包括该并集占用图在全局占用图中的坐标值时，将该N个patch的占用图在对应点云的占用图中的最大坐标值或最小坐标值，作为该并集占用图在全局占用图中的坐标值。例如，将该N个patch的占用图的u0的最小值或最大值作为该并集占用图的u0；和/或将该N个patch的占用图的v0的最小值或最大值作为该并集占用图的v0。
和/或，当该并集占用图的辅助信息包括该并集占用图在全局占用图中的尺寸信息时，将该N个patch的占用图在对应点云的占用图中的最大尺寸的信息作为该并集占用图在全局占用图中的尺寸信息。例如，将该N个patch的占用图的sizeU0的最大值作为并集占用图的sizeV0，和/或将该N个patch的占用图的sizeU0的最大值作为并集占用图的sizeV0。
和/或，当该并集占用图的辅助信息包括该并集占用图对应在三维空间中的坐标值时，将该N个patch在对应点云中的最大坐标值或最小坐标值作为该并集占用图对应在三维空间中的坐标值。例如，以最小坐标值为例，将该N个patch的u1中的最小值作为该并集占用图的u1的值，将该N个patch的v1中的最小值作为该并集占用图的v1的值，该N个patch的d1中的最小值作为该并集占用图的d1的值。
其中，一个patch的占用图在对应点云的占用图中的坐标轴，是该patch的占用图在该patch所属的点云的占用图中的坐标值。一个patch在对应点云中的坐标值，是该patch在该patch所属的点云中的坐标值。
在一种可能的设计中，该方法还包括：根据上述N帧点云中的多个patch(如所有patch)的辅助信息，重构上述N帧点云。其中，该多个patch包括上述N个patch。
在一种可能的设计中，该点云组是帧组(group of frame，GOF)GOF或子GOF， 子GOF包括一个GOF中连续的多帧点云。
在一种可能的设计中，若该点云组是子GOF，上述语法元素还包括表示该子GOF包含的点云(如每帧点云)在该子GOF所属的GOF中的位置的信息，例如该子GOF的起始帧的编号和/或该子GOF所包含的帧数等。
第二方面，提供一种点云解码方法，包括：解析码流，以得到语法元素，该语法元素包括点云组的并集占用图的辅助信息，该点云组包括N帧点云，N≥2，N是整数，该并集占用图是N帧点云中具有匹配关系的N个patch的占用图的并集；根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息；基于该N帧点云中的多个patch的辅助信息，重构该N帧点云；其中，该多个patch包括该N个patch。该技术方案的执行主体可以是解码器。可选的，该方法还可以包括显示经重构的点云。
在一种可能的设计中，并集占用图的辅助信息包括以下信息中的至少一种：并集占用图在全局占用图中的坐标值、并集占用图在全局占用图中的尺寸信息和并集占用图对应在三维空间中的坐标值。其中，全局占用图是对点云组中的多个并集占用图进行打包得到的占用图。
在一种可能的设计中，该语法元素还包括：该N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值。该情况下，根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息，包括：根据解析得到的该并集占用图的辅助信息，以及该N个patch中的一个或多个patch的对应辅助信息与并集占用图的辅助信息的差值，确定该N个patch的辅助信息。例如，将该并集占用图的辅助信息以及该N个patch的其中一个patch的对应辅助信息与并集占用图的辅助信息的差值求和，得到该patch的对应辅助信息。
在一种可能的设计中，该点云组是GOF或子GOF。子GOF包括一个GOF中连续的多帧点云。
在一种可能的设计中，若该点云组是子GOF，上述语法元素还包括：表示子GOF包含的每帧点云在子GOF所属的GOF中的位置的信息，例如该子GOF的起始帧的编号和/或该子GOF所包含的帧数等。该情况下，上述方法还包括：根据解析得到的表示该子GOF包含的每帧点云在该子GOF所属的GOF中的位置的信息，确定该子GOF包含的每帧点云在该子GOF所属的GOF中的位置。
第二方面或第二方面提供的任意一种可能的设计与第一方面或第一方面对应的可能的设计相对应，因此，第二方面或第二方面提供的任意一种可能的设计中的相关内容的解释、具体实现方式，以及有益效果的描述均可以参考上述第一方面或第一方面可能的设计中的描述，或根据上述第一方面或第一方面可能的设计中的描述推理得到，此处不再赘述。
第三方面，提供一种点云编码方法，包括：确定指示信息，并将该指示信息编入码流。该指示信息指示是否采用目标点云编码方法对点云组中的N个patch的辅助信息进行编码，该目标点云编码方法包括上述第一方面或第一方面提供的任意一种可能的设计所描述的点云编码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。也就是说，本申请实施例提供的技术方案支持从候选的至少两种点云编码方法中选择出目标点云编码方法的技术方案。
第四方面，提供一种点云解码方法，包括：解析码流，以得到指示信息，该指示信息指示是否采用目标点云解码方法对点云组中的N个patch的辅助信息进行解码，该目标点云解码方法包括上述第二方面或第二方面提供的任意一种可能的设计所描述点云解码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch；当该指示信息指示采用目标点云解码方法进行解码时，采用目标点云解码方法对该N个patch的辅助信息进行处理。也就是说，本申请实施例提供的技术方案支持从候选的至少两种点云解码方法中选择出目标点云解码方法的技术方案。
第五方面，提供一种编码器，包括：用于执行上述第一方面或第一方面任意一种可能的设计提供的方法的各个模块，或者用于执行上述第三方面或第三方面任意一种可能的设计提供的方法的各个模块。
在一种可能的实现方式中，可以根据上述第一/三方面或第一/三方面中任意一种可能的实现方式所提供的方法对编码器进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。
在一种可能的实现方式中，该编码器包括打包模块和辅助信息编码模块。
例如，应用于上述第一方面的技术方案时，打包模块用于获取点云组的并集占用图的辅助信息；该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；该并集占用图的辅助信息用于确定该N个patch的辅助信息。辅助信息编码模块用于将包含该并集占用图的辅助信息的语法元素编入码流。
又如，应用于上述第三方面的技术方案时，打包模块用于确定指示信息，该指示信息指示是否采用目标点云编码方法对点云组中的N个patch的辅助信息进行编码，该目标点云编码方法包括上述第一方面或第一方面提供的任意一种可能的设计所描述的点云编码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。辅助信息编码模块用于将该指示信息编入码流。
第六方面，提供一种解码器，包括：用于执行上述第二方面或第二方面任意一种可能的设计提供的方法的各个模块，或者用于执行上述第四方面或第四方面任意一种可能的设计提供的方法的各个模块。
在一种可能的实现方式中，可以根据上述第二/四方面或第二/四方面中任意一种可能的实现方式所提供的方法对解码器进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。
在一种可能的实现方式中，该解码器包括辅助信息解码模块和点云重构模块。
例如，应用于上述第二方面的技术方案时，辅助信息解码模块用于解析码流，以得到语法元素，该语法元素包括该点云组的并集占用图的辅助信息，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息。点云重构模块用于基于该N帧点云中的多个patch的辅助信息，重构该N帧点云；其中，该多个patch包括该N个patch。
又如，应用于上述第四方面的技术方案时，辅助信息解码模块用于解析码流，以得到指示信息，该指示信息指示是否采用目标点云解码方法对点云组中的N个patch 的辅助信息进行解码，该目标点云解码方法包括上述第二方面或第二方面提供的任意一种可能的设计所描述点云解码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch；以及，当该指示信息指示采用目标点云解码方法进行解码时，采用目标点云解码方法对该N个patch的辅助信息进行解码。点云重构模块用于基于解码得到的该N帧点云中的多个patch的辅助信息，重构所述N帧点云；该多个patch包括该N个patch。
第七方面，提供一种用于编码点云数据的设备，该设备可以包括：
存储器，用于存储点云数据。
编码器，用于执行上述第一方面或第一方面的任意一种可能的设计，或者第三方面或第三方面的任意一种可能的设计所描述的点云编码方法。
第八方面，提供一种编码装置，该装置包括存储器和处理器。存储器用于存储程序代码；处理器用于调用该程序代码，以执行第一方面或第一方面的任意一种可能的设计，或者第三方面或第三方面的任意一种可能的设计所描述的点云编码方法。
第九方面，提供一种用于解码点云数据的设备，该设备可以包括：
存储器，用于存储点云数据。
解码器，用于执行上述第二方面或第二方面的任意一种可能的设计，或者第四方面或第四方面的任意一种可能的设计所描述的点云解码方法。
第十方面，提供一种解码装置，该装置包括存储器和处理器。存储器用于存储程序代码；处理器用于调用该程序代码，以执行第二方面或第二方面的任意一种可能的设计，或者第四方面或第四方面的任意一种可能的设计所描述的点云解码方法。
第十一方面，提供一种计算机可读存储介质，该计算机可读存储介质存储了程序代码，当该程序代码在计算机上运行时，使得该计算机执行第一方面或第一方面的任一种可能的设计，或者第三方面或第三方面的任一种可能的设计所描述的点云编码方法的部分或全部步骤。
第十二方面，提供一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得该计算机执行第一方面或第一方面的任意一种可能的设计，或者第三方面或第三方面的任意一种可能的设计所描述的点云编码方法的部分或全部步骤。
第十三方面，提供一种计算机可读存储介质，该计算机可读存储介质存储了程序代码，当该程序代码在计算机上运行时，使得该计算机执行第二方面或第二方面的任一种可能的设计，或者第四方面或第四方面的任一种可能的设计所描述的点云解码方法的部分或全部步骤。
第十四方面，提供一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得该计算机执行第二方面或第二方面的任意一种可能的设计，或者第四方面或第四方面的任意一种可能的设计所描述的点云解码方法的部分或全部步骤。
应当理解的是，上述提供的相关装置/设备/计算机可读存储介质/计算机程序产品的有益效果均可以对应参考对应方面提供的方法实施例的有益效果，此处不再赘述。
应当理解的是，本申请实施例中的术语“至少一个(种/帧)”包括一个(种帧)或多个(种帧)。“多个(种帧)”是指两个(种帧)或两个(种帧)以上。例如，A、B和C中的至少一种，包括：单独存在A、单独存在B、同时存在A和B、同时存在A 和C、同时存在B和C，以及同时存在A、B和C。本申请实施例中的术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。本申请实施例中的术语字符“/”，一般表示前后关联对象是一种“或”的关系。在公式中，字符“/”表示除法运算，如A/B表示A除以B。
附图说明
图1为可适用于本申请实施例的一种实例的打包方法的流程示意图；
图2和图3为基于图1所示的打包方法的一个示例过程示意图；
图4为可适用于本申请实施例的一种实例的点云译码系统的示意性框图；
图5为可用于本申请实施例的一种实例的编码器的示意性框图；
图6为可适用于本申请实施例的一种实例的点云、patch以及点云的占用图的示意图；
图7为可用于本申请实施例的一种实例的解码器的示意性框图；
图8为本申请实施例提供的一种点云编码方法的流程示意图；
图9为本申请实施例提供的一种点云解码方法的流程示意图；
图10为本申请实施例提供的另一种点云编码方法的流程示意图；
图11为本申请实施例提供的另一种点云解码方法的流程示意图；
图12为本申请实施例提供的一种编码器的示意性框图；
图13为本申请实施例提供的一种解码器的示意性框图；
图14为本申请实施例提供的一种译码设备的示意性框图。
具体实施方式
为了便于理解本申请实施例提供的技术方案，以下对本申请实施例涉及的技术及术语进行说明。
在动态图像专家组(moving picture expert group，MPEG)点云编码方法中，编码器首先将待编码点云(即当前帧或者当前帧点云)按照一定准则分割成若干个patch，这些patch相互之间没有交叠区域。然后，将每个patch从三维空间投影到二维平面，得到一个二维图像(即patch的占用图)。接着，将所有patch的占用图(或者降低分辨率后的patch的占用图)按照某种规则紧密的排列在一张二维图像上，得到当前帧占用图。这种排列patch的占用图的方法称为打包(packing)。具体的，将该点云的各patch按照特定的顺序进行排列例如按照各patch的占用图的宽/高降序排列；然后，按照排列后的各patch的顺序，依次将patch的占用图插入该点云占用图的可用区域中，得到该点云的占用图。后续，按照打包顺序生成当前帧深度图和当前帧纹理图。其中，当前帧深度图是各patch经过投影得到的深度按照打包顺序产生的一张二维图像。当前帧纹理图是各patch经过投影得到的纹理图按照打包顺序产生的一张二维图像。当前帧占用图是一张2值二维图像，用于表示该二维图像的各像素位置是否被点云中某点占据。通常，为了节约编码比特，当前帧占用图的分辨率低于当前帧深度图和当前帧纹理图的分辨率。为了方便描述，下文中均以不考虑分辨率是否降低为例进行说明。
为了提高编码或压缩效率，一种打包方法如图1所示，图1所示的方法包括如下步骤：
步骤1：获取点云组的全局匹配patch。该点云组包括N帧点云。N≥2，N是整数。
其中，一帧点云的全局匹配patch是指该帧点云中的且在点云组中除该帧点云之外的各帧点云中均能找到具有匹配关系的patch的patch。同时，与该patch具有匹配关系的patch是全局匹配patch。其中，与一个patch具有匹配关系的patch是目标patch的匹配patch，目标patch是该patch或者该patch的匹配patch。可选的，一帧点云中的一个patch在另一帧点云中的匹配patch，可以是该另一帧点云中的与该patch的交并比(intersection over union，IoU)最大且该IoU大于或等于预设阈值的patch。
对于任一全局匹配patch来说，与该全局匹配patch具有匹配关系的patch的个数是N-1。
步骤2：确定与M个集合对应的M个并集占用图，每个集合包括N个全局匹配patch，该N个全局匹配patch是N帧点云中具有匹配关系的patch(或跨N帧点云中具有匹配关系的patch)。可选的，第m个集合对应的并集占用图是第m个集合中的各全局匹配patch的占用图的并集；1≤m≤M，m和M均是整数。
其中，并集占用图也可以称为并集占用图。
一个集合包括的N个全局匹配patch是跨N帧点云中具有匹配关系的patch，可以理解为：该N个全局匹配patch中的每个全局匹配patch来自于(或所属)一帧点云、不同全局匹配patch来自于不同点云、且该N个全局匹配patch具有匹配关系。
一个全局匹配patch所属的集合对应的并集占用图覆盖该全局匹配patch的占用图。
可选的，编码器在执行步骤2之后，该方法还可以包括：
步骤2a：获取并集占用图的辅助信息。其中，关于并集占用图的辅助信息的具体描述，以及获得并集占用图的辅助信息的具体实现方式等均可以参考下文，此处不再赘述。当然，本申请实施例对步骤2a与步骤3～4的顺序不进行限定。
步骤3：对上述M个并集占用图进行打包，得到全局占用图(即最终的占用图)。其中，全局占用图用于确定M个并集占用图在全局占用图中的位置。其中，打包的过程可以认为是对全局占用图进行更新的过程，如将这M个并集占用图按照从上到下，从左到右的顺序排列在全局占用图的可用空间，得到最终的占用图。具体方法可以参考现有技术，此处不再赘述。
步骤4：对N帧点云中的每帧点云进行打包，得到N帧点云的占用图。第n帧点云中的第m个全局匹配patch的占用图在第n帧点云的占用图中的位置(第一位置)，对应于M个并集占用图中的第m个并集占用图在全局占用图中的位置(第二位置)；1≤n≤N，n是整数。
其中，第一位置对应于第二位置可以包括：第一位置的坐标值与第二位置的坐标值相同；或者第一位置的坐标值在其坐标系下与第二位置的坐标值在其坐标系下实质性相同；或者第一位置的坐标值与第二位置的坐标值不相同，但第二位置所在的位置范围覆盖第一位置所在的位置范围。其中，第一位置的坐标值可通过第n帧点云的占用图中的第m个全局匹配patch的占用图在第n帧点云的占用图中的位置坐标来表征。第二位置的坐标值可通过第m个并集占用图在全局占用图中的位置坐标来表征。第一位置所在的位置范围是第m个全局匹配patch的占用图所占的区域，第二位置所在的位置范围是第m个并集占用图所占的区域。
以下通过一个简单的示例，对图1中的打包方法进行说明。
图2和图3为基于图1提供的打包方法的一个示例的过程示意图。图2和图3中，假设点云组包括4帧点云，每帧点云包括10个patch，且全局匹配patch分别是：点云1中的patch11、patch12和patch13；点云2中的patch21、patch22和patch23；点云3中的patch31、patch32和patch33；点云4中的patch41、patch42和patch43。并且，patch11、patch21、patch31和patch41具有匹配关系，这些patch组成集合1。patch12、patch22、patch32和patch42具有匹配关系，这些patch组成集合2。patch13、patch23、patch33和patch43具有匹配关系，这些patch组成集合3。也就是说，上述M个集合具体是集合1～3，且每个集合包含4个具有匹配关系的patch。
基于该示例得到的全局占用图可以如图2所示。图2中的较大矩形表示全局占用图，图2中的椭圆、三角形和较小矩形分别表示集合1～3对应的并集patch占用图。
基于该示例，执行上述步骤4之后，点云1的占用图可以如图3中的(a)所示，点云2的占用图可以如图3中的(b)所示。图3仅示意出了点云1的占用图和点云2的占用图中的部分patch的占用图，且没有示意出点云3和点云4的占用图。
对比图2和图3可知，patch11的占用图在点云1的占用图中的位置和patch21的占用图在点云2的占用图中的位置均对应并集patch占用图1在全局占用图中的位置；patch12的占用图在点云1的占用图中的位置和patch22的占用图在点云2的占用图中的位置均对应并集patch占用图2在全局占用图中的位置；patch13的占用图在点云1的占用图中的位置和patch23的占用图在点云2的占用图中的位置均对应并集patch占用图3在全局占用图中的位置。
作为一个示例，关于图1所示的技术方案中的相关内容的解释以及具体实现方式的说明，可以例如但不限于参考申请人其他的在先申请的专利。
按照传统的点云编码方法，编码器需要将一帧点云中的各patch的信息编入码流。这样，会导致较大的码流传输开销。为此，本申请实施例提供了点云编解码方法和编解码器，用以在不影响编解码性能的前提下，节省码流传输开销。
为了说明patch的信息，首先说明点云译码系统中定义的坐标系。对于帧级别占用图(二维图像)来说，通常定义横轴是U0轴，纵轴是V0轴。对于点云(三维空间)来说，通常定义横轴为U1轴、纵轴是V1轴，垂直于U1轴和V1轴的坐标轴是D1轴。也就是说，U1轴是世界坐标系中的X轴，V1轴是世界坐标系中的Y轴，D轴是世界坐标系中的Z轴。
patch的信息的示例以及描述可以参考以下表1：
表1
Figure PCTCN2019103293-appb-000001
基于表1，当patch级别的占用图是点云中的patch的占用图时，帧级别的占用图可以是该点云的占用图。当patch级别的占用图是并集占用图时，帧级别的占用图可以是全局占用图。下文涉及到的patch的信息的解释均可以参考表1，在此统一说明，下文不再赘述。
以下，说明可用于本申请实施例的系统架构的示意性框图和装置的示意性框图。
图4为可用于本申请实施例的一种实例的点云译码系统1的示意性框图。术语“点云译码”或“译码”可一般地指代点云编码或点云解码。点云译码系统1的编码器100可以根据本申请提出的任一种点云编码方法对待编码点云进行编码。点云译码系统1的解码器200可以根据本申请提出的与编码器使用的点云编码方法相对应的点云解码方法对待解码点云进行解码。
如图4所示，点云译码系统1包含源装置10和目的地装置20。源装置10产生经编码点云数据。因此，源装置10可被称为点云编码装置。目的地装置20可对由源装置10所产生的经编码的点云数据进行解码。因此，目的地装置20可被称为点云解码装置。源装置10、目的地装置20或两个的各种实施方案可包含一或多个处理器以及耦合到所述一或多个处理器的存储器。所述存储器可包含但不限于随机存取存储器(random access memory，RAM)、只读存储器(read-only memory，ROM)、带电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、快闪存储器或可用于以可由计算机存取的指令或数据结构的形式存储所要的程序代码的任何其它媒体，如本文所描述。
源装置10和目的地装置20可以包括各种装置，包含桌上型计算机、移动计算装置、笔记型(例如，膝上型)计算机、平板计算机、机顶盒、例如所谓的“智能”电话等电话手持机、电视机、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机或其类似者。
目的地装置20可经由链路30从源装置10接收经编码点云数据。链路30可包括能够将经编码点云数据从源装置10移动到目的地装置20的一或多个媒体或装置。在一个实例中，链路30可包括使得源装置10能够实时将经编码点云数据直接发送到目的地装置20的一或多个通信媒体。在此实例中，源装置10可根据通信标准(例如无 线通信协议)来调制经编码点云数据，且可将经调制的点云数据发送到目的地装置20。所述一或多个通信媒体可包含无线和/或有线通信媒体，例如射频(radio frequency，RF)频谱或一或多个物理传输线。所述一或多个通信媒体可形成基于分组的网络的一部分，基于分组的网络例如为局域网、广域网或全球网络(例如，因特网)。所述一或多个通信媒体可包含路由器、交换器、基站或促进从源装置10到目的地装置20的通信的其它设备。
在另一实例中，可将经编码数据从输出接口140输出到存储装置40。类似地，可通过输入接口240从存储装置40存取经编码点云数据。存储装置40可包含多种分布式或本地存取的数据存储媒体中的任一者，例如硬盘驱动器、蓝光光盘、数字多功能光盘(digital versatile disc，DVD)、只读光盘(compact disc read-only memory，CD-ROM)、快闪存储器、易失性或非易失性存储器，或用于存储经编码点云数据的任何其它合适的数字存储媒体。
在另一实例中，存储装置40可对应于文件服务器或可保持由源装置10产生的经编码点云数据的另一中间存储装置。目的地装置20可经由流式传输或下载从存储装置40存取所存储的点云数据。文件服务器可为任何类型的能够存储经编码的点云数据并且将经编码的点云数据发送到目的地装置20的服务器。实例文件服务器包含网络服务器(例如，用于网站)、文件传输协议(file transfer protocol，FTP)服务器、网络附属存储(network attached storage，NAS)装置或本地磁盘驱动器。目的地装置20可通过任何标准数据连接(包含因特网连接)来存取经编码点云数据。这可包含无线信道(例如，Wi-Fi连接)、有线连接(例如，数字用户线路(digital subscriber line，DSL)、电缆调制解调器等)，或适合于存取存储在文件服务器上的经编码点云数据的两者的组合。经编码点云数据从存储装置40的传输可为流式传输、下载传输或两者的组合。
图4中所说明的点云译码系统1仅为实例，并且本申请的技术可适用于未必包含点云编码装置与点云解码装置之间的任何数据通信的点云译码(例如，点云编码或点云解码)装置。在其它实例中，数据从本地存储器检索、在网络上流式传输等等。点云编码装置可对数据进行编码并且将数据存储到存储器，和/或点云解码装置可从存储器检索数据并且对数据进行解码。在许多实例中，由并不彼此通信而是仅编码数据到存储器和/或从存储器检索数据且解码数据的装置执行编码和解码。
在图4的实例中，源装置10包含数据源120、编码器100和输出接口140。在一些实例中，输出接口140可包含调节器/解调器(调制解调器)和/或发送器(或称为发射器)。数据源120可包括点云捕获装置(例如，摄像机)、含有先前捕获的点云数据的点云存档、用以从点云内容提供者接收点云数据的点云馈入接口，和/或用于产生点云数据的计算机图形系统，或点云数据的这些来源的组合。
编码器100可对来自数据源120的点云数据进行编码。在一些实例中，源装置10经由输出接口140将经编码点云数据直接发送到目的地装置20。在其它实例中，经编码点云数据还可存储到存储装置40上，供目的地装置20以后存取来用于解码和/或播放。
在图4的实例中，目的地装置20包含输入接口240、解码器200和显示装置220。在一些实例中，输入接口240包含接收器和/或调制解调器。输入接口240可经由链路 30和/或从存储装置40接收经编码点云数据。显示装置220可与目的地装置20集成或可在目的地装置20外部。一般来说，显示装置220显示经解码点云数据。显示装置220可包括多种显示装置，例如，液晶显示器(liquid crystal display，LCD)、等离子显示器、有机发光二极管(organic light-emitting diode，OLED)显示器或其它类型的显示装置。
尽管图4中未图示，但在一些方面，编码器100和解码器200可各自与音频编码器和解码器集成，且可包含适当的多路复用器-多路分用器(multiplexer-demultiplexer，MUX-DEMUX)单元或其它硬件和软件，以处置共同数据流或单独数据流中的音频和视频两者的编码。在一些实例中，如果适用的话，那么MUX-DEMUX单元可符合ITU H.223多路复用器协议，或例如用户数据报协议(user datagram protocol，UDP)等其它协议。
编码器100和解码器200各自可实施为例如以下各项的多种电路中的任一者：一个或多个微处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)、离散逻辑、硬件或其任何组合。如果部分地以软件来实施本申请，那么装置可将用于软件的指令存储在合适的非易失性计算机可读存储媒体中，且可使用一或多个处理器在硬件中执行所述指令从而实施本申请技术。前述内容(包含硬件、软件、硬件与软件的组合等)中的任一者可被视为一或多个处理器。编码器100和解码器200中的每一者可包含在一或多个编码器或解码器中，所述编码器或解码器中的任一者可集成为相应装置中的组合编码器/解码器(编码解码器)的一部分。
本申请可大体上将编码器100称为将某些信息“发信号通知”或“发送”到例如解码器200的另一装置。术语“发信号通知”或“发送”可大体上指代用以对经压缩点云数据进行解码的语法元素和/或其它数据的传送。此传送可实时或几乎实时地发生。替代地，此通信可经过一段时间后发生，例如可在编码时在经编码位流中将语法元素存储到计算机可读存储媒体时发生，解码装置接着可在所述语法元素存储到此媒体之后的任何时间检索所述语法元素。
如图5所示，为可用于本申请实施例的一种实例的编码器100的示意性框图。图5是以MPEG(Moving Picture Expert Group)点云压缩(Point Cloud Compression，PCC)编码框架为例进行说明的。在图5的实例中，编码器100可以包括patch信息生成模块101、打包模块102、深度图生成模块103、纹理图生成模块104、第一填充模块105、基于图像或视频的编码模块106、占用图编码模块107、辅助信息编码模块108和复用模块109等。另外，编码器100还可以包括点云滤波模块110、第二填充模块111和点云重构模块112等。其中：
patch信息生成模块101，用于采用某种方法将一帧点云分割产生多个patch，以及获得所生成的patch的相关信息等。其中，patch是指一帧点云中部分点构成的集合，通常一个连通区域对应一个patch。patch的相关信息可以包括但不限于以下信息中的至少一项：点云所分成的patch的个数、patch在三维空间中的位置信息、patch的法线坐标轴的索引、patch从三维空间投影到二维空间产生的深度图、patch的深度图大小(例如深度图的宽和高)、patch从三维空间投影到二维空间产生的占用图等。该相关 信息中的部分，如点云所分成的patch的个数、patch的法线坐标轴的索引、patch的深度图大小、patch在点云中的位置信息、patch的占用图的尺寸信息等，可以作为辅助信息被发送到辅助信息编码模块108，以进行编码(即压缩编码)。每个patch的占用图可以被发送到打包模块102进行打包。再一方面，各patch在该点云占用图中的具体位置信息和各patch的深度图等可以被发送到深度图生成模块103。
打包模块102获得该点云的占用图后，一方面可以将该点云的占用图经第二填充模块111进行填充后发送到占用图编码模块107以进行编码。另一方面可以利用该点云的占用图指导深度图生成模块103生成该点云的深度图和指导纹理图生成模块104生成该点云的纹理图。
应用于本申请实施例提供的技术方案时，作为一个示例，打包模块102还可以将打包过程中生成的并集占用图的辅助信息等发送给辅助信息编码模块108，以将并集占用图的辅助信息等编入码流，这样，不需要通过码流传输与该并集占用图对应的全局匹配patch的相关信息，因此，可以节省码流开销。关于该技术方案中的术语以及具体实现方式均可以参考下文。另外，将并集占用图的辅助信息编入码流的模块也可以是独立于辅助信息编码模块108的一个模块，本申请实施例对此不进行限定。
深度图生成模块103，用于根据点云的占用图、该点云的各patch的占用图和深度信息，生成该点云的深度图，并将所生成的深度图发送到第一填充模块105，以对深度图中的空白像素进行填充，得到经填充的深度图。
纹理图生成模块104，用于根据点云的占用图、该点云的各patch的占用图和纹理信息，生成该点云的纹理图，并将所生成的纹理图发送到第一填充模块105，以对纹理图中的空白像素进行填充，得到经填充的纹理图。
经填充的深度图和经填充的纹理图被第一填充模块105发送到基于图像或视频的编码模块106，以进行基于图像或视频的编码。后续：
一方面，基于图像或视频的编码模块106、占用图编码模块107、辅助信息编码模块108，将所得到的编码结果(即码流)发送到复用模块109，以合并成一个码流，该码流可以被发送到输出接口140。
另一方面，基于图像或视频的编码模块106所得到的编码结果(即码流)发送到点云重构模块112进行点云重构，以得到经重构的点云(即得到重构的点云几何信息)。具体的，对基于图像或视频的编码模块106所得到的经编码的深度图进行视频解码，获得点云的解码深度图，利用解码深度图、该点云的占用图和各patch的辅助信息，获得重构的点云几何信息。其中，点云的几何信息是指点云中的点(例如点云中的每个点)在三维空间中的坐标值。
可选的，点云重构模块112还可以将点云的纹理信息和重构的点云几何信息发送到着色模块，着色模块用于对重构点云进行着色，以获得重构点云的纹理信息。
可选的，纹理图生成模块104还可以基于经点云滤波模块110对重构的点云几何信息进行滤波得到的信息，生成该点云的纹理图。
如图6所示，为可适用于本申请实施例的一种实例的点云、patch以及点云的占用图的示意图。其中，图6中的(a)为一帧点云的示意图，图6中的(b)为基于图6中的(a)获得的点云的patch的示意图，图6中的(c)为图6中的(b)所示的各patch 映射到二维平面上所得到的各patch的占用图经打包得到的该点云的占用图的示意图。其中，图6中的(c)所示的点云的占用图，可以是依据上文中图1所示的方法得到的。
如图7所示，为可用于本申请实施例的一种实例的解码器200的示意性框图。其中，图7中是以MPEG PCC解码框架为例进行说明的。在图7的实例中，解码器200可以包括解复用模块201、基于图像或视频的解码模块202、占用图解码模块203、辅助信息解码模块204、点云重构模块205、点云滤波模块206和点云的纹理信息重构模块207。其中：
解复用模块201用于将输入的码流(即合并的码流)发送到相应解码模块。具体的，将包含经编码的纹理图的码流和经编码的深度图的码流发送给基于图像或视频的解码模块202；将包含经编码的占用图的码流发送给占用图解码模块203，将包含经编码的辅助信息的码流发送给辅助信息解码模块204。
基于图像或视频的解码模块202，用于对接收到的经编码的纹理图和经编码的深度图进行解码；然后，将解码得到的纹理图信息发送给点云的纹理信息重构模块207，将解码得到的深度图信息发送给点云重构模块205。占用图解码模块203，用于对接收到的包含经编码的占用图的码流进行解码，并将解码得到的占用图信息发送给点云重构模块205。辅助信息解码模块204，用于对接收到的经编码的辅助信息进行解码，并将解码得到的指示辅助信息的信息发送给点云重构模块205。
点云重构模块205，用于根据接收到的占用图信息和辅助信息对点云的几何信息进行重构。经重构的点云的几何信息经点云滤波模块206滤波之后，被发送到点云的纹理信息重构模块207。作为一个示例，点云重构模块205执行的重构过程可以参考上文中点云重构模块112所执行的重构过程。
点云的纹理信息重构模块207，用于对点云的纹理信息进行重构，得到经重构的点云。
可以理解的，图7所示的解码器200仅为示例，具体实现时，解码器200可以包括比图7中所示的更多或更少的模块。本申请实施例对此不进行限定。
以下，结合附图对本申请实施例提供的点云编解码方法和编解码器进行说明。
如图8所示，为本申请实施例提供的一种点云编码方法的流程示意图。作为一个示例，结合图4，图8所示的方法的执行主体可以是编码器100。图8所示的方法包括如下步骤：
S101：获取点云组的并集占用图的辅助信息；其中，该点云组包括N帧点云，N≥2，N是整数，并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；并集占用图的辅助信息用于确定该N个patch的辅助信息。例如，基于该N个patch的辅助信息，获取该并集占用图的辅助信息。
点云组包括至少两帧点云，例如在时间上连续的至少两帧点云。
可选的，点云组可以是一个或多个GOF，通常是指一个GOF。GOF中点云的帧数是可配置的。本申请实施例对一个GOF所包含的点云的帧数不进行限定，例如，一个GOF可以包含32帧点云。关于GOF的确定方式可以参考现有技术。
可选的，点云组可以是一个子GOF，该子GOF可以由一个GOF中的时间上连续的多帧点云构成。一个GOF可以包括至少两个子GOF，该至少两个子GOF中的任意 两个子GOF所包含的点云的帧数可以相等，也可以不相等。
具体实现的过程中，编码器和解码器可以基于同一预定义(如通过协议预定义)方式将一个GOF分为至少两个子GOF。或者，编码器可以将一个GOF分为至少两个子GOF(如根据GOF中包含的点云间的内容之间的连续性将一个GOF分为至少两个子GOF)，然后将表示该至少两个子GOF在该GOF中的位置的信息(即每个子GOF包括哪些点云的信息)编入码流，以使得解码器通过解析码流得到的信息获取该至少两个子GOF在该GOF中的位置。
可选的，表示该至少两个子GOF在该GOF中的位置的信息可以包括：每个子GOF的起始帧(即第一帧点云)的编号和/或每个子GOF所包含的帧数等。可以理解的，第一个子GOF的起始帧的编号也可以不被编入码流和/或其中一个GOF所包含的帧数也可以不被编入码流。该可选的实施例可以应用于该至少两个子GOF中的每个子GOF包括的帧数均相同的场景中，也可以应用于该至少两个子GOF中的部分或全部子GOF包括的帧数不同的场景中。
也就是说，在一种可选的实现方式，当点云组是子GOF时，上述语法元素还可以包括表示子GOF中的每帧点云在该子GOF所属的GOF中的位置的信息，例如，该子GOF包含的起始帧的编号和/或该子GOF包含的点云的帧数。
可选的，如果该至少两个子GOF中的每个子GOF包括的帧数均相同，则表示该至少两个子GOF在该GOF中的位置的信息可以包括：一个子GOF包括的点云的帧数，或一个GOF包含的子GOF的个数。
点云组的并集占用图的数量可以是一个或多个，例如结合图1所示的打包方法，点云组的并集占用图的数量是M。S101中的并集占用图可以是点云组中的任意一个并集占用图。关于并集占用图、具有匹配关系的patch等相关术语的描述可以参考上文，此处不再赘述。
S101中的N个patch中的每个patch可以认为是上文中所描述的全局匹配patch。该N个patch中的每个patch来自一帧点云，且不同patch来自不同点云。关于全局匹配patch的相关解释可以参考上文，此处不再赘述。
并集占用图的辅助信息，是指用于描述该并集占用图的信息，例如但不限于包括以下信息中的至少一种：该并集占用图在全局占用图中的坐标值、该并集占用图在全局占用图中的尺寸信息和该并集占用图对应在三维空间中的坐标值。全局占用图是对点云组中的多个并集占用图进行打包得到的占用图。关于全局占用图的相关描述可以参考上文，此处不再赘述。
其中，并集占用图在全局占用图中的坐标值，包括：并集占用图在全局占用图中的U0轴上的坐标值(如并集占用图的u0)，和/或V0轴上的坐标值(如并集占用图的v0)。具体实现时，编码器可以将并集占用图对应的N个patch的占用图的u0的最小值或最大值等作为该并集占用图的u0；和/或将并集占用图对应的N个patch的占用图的v0的最小值或最大值等作为该并集占用图的v0。
其中，并集占用图在全局占用图中的尺寸信息，包括：并集占用图在全局占用图中的U0轴上的尺寸信息(如并集占用图的sizeU0)，和/或V0轴上的尺寸信息(如并集占用图的sizeV0)。具体实现时，编码器可以将该并集占用图对应的N个patch 的占用图的sizeU0的最大值作为并集占用图的sizeU0，和/或将该并集占用图对应的N个patch的占用图的sizeU0的最大值作为并集占用图的sizeV0。
其中，并集占用图对应在三维空间中的坐标值，是指该并集占用图对应的N个patch在三维空间中的最小(或最大等)坐标值。以最小坐标值为例，并集占用图对应在三维空间中的坐标值可以包括：该N个patch在三维空间中的U1轴上的最小坐标值(如最小u1值)、在V1轴上的最小坐标值(如最小v1值)，和/或在D1轴上的最小坐标值(如最小d1值)。
在一种实现场景中，S101中获取并集占用图的辅助信息的步骤可以在编码器获得并集占用图之后得到，例如，结合图1所示的打包方法，可以在执行打包方法的过程中得到，当然也可以在执行打包方法之后得到，本申请实施例对此不进行限定。
并集占用图的辅助信息用于确定该N个patch的辅助信息，可以理解为：并集占用图的辅助信息用于解码器确定该N个patch的辅助信息。
为了便于描述，本申请实施例中，将S101中的N个patch称为S101中的并集占用图对应的N个patch。在此统一说明，下文不再赘述。
S102：将语法元素编入码流，该语法元素包括该并集占用图的辅助信息。
上述S101～S102是从一个并集占用图对应的N个patch的角度说明点云编码方法的，具体实现时，对于点云组中的任意一个或多个并集占用图对应的patch(即全局匹配patch)均可以基于S101～S102执行点云编码方法。对于点云组中的其他patch，本申请实施例对其所适用的点云编码方法不进行限定，例如可以按照现有技术提供的任意一种点云编码方法如MPEG点云编码方法进行编码。该其他patch可以包括全部的非全局匹配patch，进一步可以包括除按照上述S101～S102执行点云编码方法的全局匹配patch之外的其他全局匹配patch。
具体实现的过程中，作为一个示例，编码器可以在针对一个并集占用图对应的N个patch执行S101～S102之后，再针对另一个并集占用图对应的N个patch执行S101～102，直至执行完所有需要采用本申请实施例提供的技术方案进行编码的并集占用图对应的patch。作为另一个示例，编码器可以在依次或同时针对需要采用本申请实施例提供的技术方案进行编码的并集占用图对应的patch，执行上述S101之后，再依次或同时针对这并集占用图对应的patch执行上述S102，本申请实施例对此不进行限定。
可以理解的，由于并集占用图与该并集占用图对应的N个patch的占用图之间的相关性，例如并集占用图在全局占用图中的位置信息，对应于上述N个patch的占用图在其所属的点云的占用图中的位置信息，和/或并集占用图是该N个patch的占用图的并集等，因此，可以通过并集占用图的辅助信息表示或近似表示该N个patch的辅助信息。
本申请实施例提供的点云编码方法中，编码器将并集占用图的辅助信息编入码流，以使得解码器基于该并集占用图的辅助信息确定该并集占用图对应的N个patch的辅助信息。这样，相比现有技术中在码流中直接传输这N个patch的辅助信息的技术方案，本申请实施例提供的技术方案有助于在不影响编解码性能的前提下，节省码流传输开销。
可选的，图8所示的方法还包括以下步骤S103：
S103：根据上述N帧点云中的多个patch(如所有patch)的辅助信息，重构N帧点云；其中，该多个patch包括上述N个patch。S103的具体实现方式可以参考上文，此处不再赘述。
本申请实施例对S101～S102和S103的执行顺序不进行限定，例如，先执行S101～S102再执行S103，或先执行S103再执行S101～S102，或执行S101～S102的过程中执行S103等。
可选的，图8所示的实施例中的语法元素还包括上述N个patch中的一个或多个patch(例如所有patch)的对应辅助信息与该并集占用图的辅助信息的差值。
其中，该一个或多个patch的对应辅助信息，是指与该并集占用图的辅助信息是同类型的辅助信息。例如，并集占用图的辅助信息和该一个或多个patch的对应辅助信息均是u0或均是v0；或者，均是sizeU0或均是sizeV0；或者，均是u1或均是d1或均是v1。
该可选的实现方式可以理解为使用差分方式传输并集占用图对应的一个或多个patch的辅助信息。虽然并集占用图与该并集占用图对应的N个patch的占用图之间的相关性，但是，并集占用图的辅助信息和该N个patch的辅助信息可能并不完全相同。因此，使用差分方式传输并集占用图对应的一个或多个patch的辅助信息，有助于提高N个patch中的该一个或多个patch的辅助信息的准确度和/或设置灵活性。例如，可以提高该一个或多个patch的u0、v0、u1、v1和/或d1的准确度。例如，可以提高该一个或多个patch的sizeU0和/或sizeV0的设置灵活性，也就是说，不必局限于该一个或多个patch的sizeU0与该并集占用图的sizeU0相等，和/或不必局限于该一个或多个patch的sizeV0与该并集占用图的sizeV0相等。
具体实现的过程中，语法元素中是否包含上述N个patch中的一个或多个patch(例如所有patch)的对应辅助信息与该并集占用图的辅助信息的差值，可以是预定义的如通过协议预定义的，也可以是编码器将指示是否包含该差值的指示信息编入码流并发送给解码器。
如图9所示，为本申请实施例提供的一种点云编码方法的流程示意图。作为一个示例，结合图4，图9所示的方法的执行主体可以是解码器200。图9所示的方法包括如下步骤：
S201：解析码流，以得到语法元素，该语法元素包括点云组的并集占用图的辅助信息，该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集。
可选的，该语法元素还包括上述N个patch中的一个或多个patch(例如所有patch)的对应辅助信息与该并集占用图的辅助信息的差值。该可选的方式与图8所示的实施例中的语法元素包括该差值的实施例相对应。
S202：根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息。
例如，对于任意一个并集占用图来说，若语法元素包括该并集占用图的辅助信息，不包括该并集占用图的对应的N个patch中的任意一个patch的对应辅助信息与该并集占用图的辅助信息的差值，则解码器可以将该并集占用图的辅助信息作为该N个patch 中的每个patch的对应辅助信息。
又如，对于任意一个并集占用图来说，若语法元素包括该并集占用图的辅助信息，且包括该并集占用图的对应的N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值，则对于该一个或多个patch中的每个patch来说，解码器可以基于该并集占用图的辅助信息与该patch的对应辅助信息之间的差值，确定该patch的对应辅助信息，具体的，对该并集占用图的辅助信息和该差值求和，得到该patch的对应辅助信息。可选的，如果该N个patch还包括除该一个或多个patch之外的其他patch，则对于该其他patch中的每个patch来说，解码器可以将该并集占用图的辅助信息作为该N个patch的对应辅助信息。
可以理解的，对于点云组中的非并集占用图对应的每个patch(即点云组中的非全局匹配patch)来说，解码器可以通过解析码流，获得该patch的辅助信息的方法可以参考现有技术。
S203：基于该N帧点云中的多个patch(例如所有patch)的辅助信息，重构该N帧点云；其中，该多个patch包括该N个patch。S203的具体实现方式可以参考上文，此处不再赘述。
本实施例提供的任一种点云解码方法与图8所示的实施例中相应的点云编码方法对应，因此，本实施例中相关术语、具体实现方式以及有益效果的描述均可以参考上述点云编码方法的实施例，此处不再赘述。例如，点云组包括GOF或子GOF等。其中，关于解码器如何获得子GOF在该子GOF所属的GOF中的位置可以参考上文，此处不再赘述。又如，关于本实施例中的N个patch的相关描述等等均可以参考上文。
如图10所示，为本申请实施例提供的一种点云编码方法的流程示意图。作为一个示例，结合图4，图10所示的方法的执行主体可以是编码器100。图10所示的方法包括如下步骤：
S301：确定指示信息，该指示信息指示是否采用目标点云编码方法对点云组中的N个patch的辅助信息进行编码。目标点云编码方法可以是本申请实施例提供的任意一种点云编码方法，例如图8所示的点云编码方法。其中，该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。
具体实现的过程中，编码器可以支持至少两种编码方法，该至少两种的其中一种可以是本申请实施例提供的任一种点云编码方法如图8所示的点云编码方法，其他其他点云编码方法可以是现有技术或未来提供的任意一种点云编码方法。
可选的，该指示信息可以是目标点云编码/解码方法的索引。具体实现的过程中，编码器和解码器可以预先约定编码器/解码器所支持的至少两种点云编码/解码方法的索引，然后，在编码器确定目标编码方法之后，将目标编码方法的索引或该目标编码方法对应的解码方法的索引作为指示信息编入码流。本申请实施例对编码器如何确定目标编码方法是编码器所支持的至少两种编码方法中的哪一种不进行限定，例如根据码流传输开销确定目标编码方法。
S302：将该指示信息编入码流。
具体实现的过程中，该方法还包括如下步骤S303：
S303：根据目标点云编码方法对该N个patch的辅助信息进行编码。例如，根据 图8所示的点云编码方法对该N个patch的辅助信息进行编码。
本实施例提供了一种选择目标编码方法的技术方案，该技术方案可以应用于编码器支持至少两种点云编码方法的场景中。也就是说，本申请实施例支持编码器从候选的至少两种点云编码方法中选择目标点云编码方法的技术方案。
如图11所示，为本申请实施例提供的一种点云解码方法的流程示意图。作为一个示例，结合图4，图11所示的方法的执行主体可以是解码器200。图11所示的方法包括如下步骤：
S401：解析码流，以得到指示信息，该指示信息指示是否采用目标点云解码方法对点云组中的N个patch的辅助信息进行解码。目标点云解码方法可以是本申请实施例提供的任意一种点云解码方法，例如图9所示的点云解码方法。其中，该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。
S402：当该指示信息指示采用目标点云解码方法进行解码时，采用目标点云解码方法对该N个patch的辅助信息进行处理。具体可以参考上述S201～S203。
本实施例提供的点云解码方法与图10提供的点云编码方法相对应。因此，本实施例中相关内容的解释、具体实现方式以及有益效果的描述均可以参考上述图10所示的实施例。本申请实施例支持编码器从候选的至少两种点云解码方法中选择目标点云解码方法的技术方案。
以下，以点云组是一个子GOF(subGOF)为例，说明实现上文提供的码流格式的示例。具体的，表示GOF中的全局匹配patch的辅助信息(group_of_frames_auxiliary_information)的码流的格式可以如表2所示：
表2
Figure PCTCN2019103293-appb-000002
其中，表2中的group_of_frames_size表示GOF中包含的点云的帧数，subGOFsize表示子GOF中包含的点云的帧数。
基于表2，表示子GOF中的全局匹配patch(即具有匹配关系的全局匹配patch)的辅 助信息(subGOF_global_matched_patch_auxiliary_information)的码流的格式可以如表3所示：
表3
Figure PCTCN2019103293-appb-000003
其中，global_matched_patch_count表示一帧中的全局匹配patch的个数。表3中是以该集合中的各全局patch的u0和d0采用差分方式传输，且u1、v1和d1均采用该集合对应的并集占用图的对应辅助信息为例进行说明的。另外，unionpatch[i].normal_axis表示并集占用图的法线坐标轴索引，可以理解的，一个并集占用图对应的所有patch的法线坐标轴相同，因此，可以使用并集占用图的法线坐标轴索引表示该所有patch的法线坐标轴。
基于表2，表示一帧中的所有patch的辅助信息(具体是指u1、v1和d1)的码流的格式可以如表4所示：
表4
Figure PCTCN2019103293-appb-000004
其中，表4中是以全局匹配patch的u1、v1和d1通过差分方式传输为例进行说明的。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对编码器和解码器进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
图12为本申请实施例提供的一种编码器120的示意性框图。编码器120可以用于执行图8或图10所示的点云编码方法。编码器120包括打包模块1201和辅助信息编码模块1202。
在一些实施例中：打包模块1201用于获取点云组的并集占用图的辅助信息；该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；该并集占用图的辅助信息用于确定该N个patch的辅助信息。辅助信息编码模块1202用于将语法元素编入码流，该语法元素包括该并集占用图的辅助信息。例如，结合图8，打包模块1201可以用于执行S101，辅助信息编码模块1202可以用于执行S102。
可选的，并集占用图的辅助信息包括以下信息中的至少一种：并集占用图在全局占用图中的坐标值、并集占用图在全局占用图中的尺寸信息和并集占用图对应在三维空间中的坐标值。全局占用图是对点云组中的多个并集占用图进行打包得到的占用图。
可选的，上述语法元素还包括：该N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值。
可选的，打包模块1201具体用于：基于该N个patch的辅助信息，获取该并集占用图的辅助信息。
可选的，当该并集占用图的辅助信息包括该并集占用图在全局占用图中的坐标值时，打包模块1201具体用于：将该N个patch的占用图在对应点云的占用图中的最大坐标值或最小坐标值，作为该并集占用图在全局占用图中的坐标值；
可选的，当该并集占用图的辅助信息包括该并集占用图在全局占用图中的尺寸信息时，打包模块1201具体用于：将该N个patch的占用图在对应点云的占用图中的最大尺寸的信息作为该并集占用图在全局占用图中的尺寸信息；
可选的，当该并集占用图的辅助信息包括该并集占用图对应在三维空间中的坐标值时，打包模块1201具体用于：将该N个patch在对应点云中的最大坐标值或最小坐标值作为该并集占用图对应在三维空间中的坐标值。
可选的，编码器120还包括点云重构模块1203，用于根据该N帧点云中的多个patch的辅助信息，重构该N帧点云；其中，该多个patch包括该N个patch。例如，结合图8，点云重构模块1203可以用于执行S103。
可选的，该点云组是GOF或子GOF，该子GOF包括一个GOF中连续的多帧点云。
可选的，若该点云组是该子GOF，上述语法元素还包括：表示该子GOF包含的点云(如每帧点云)在该子GOF所属的GOF中的位置的信息。例如，该子GOF包含的起始帧的编号，和/或该子GOF包含的点云的帧数。换言之，根据该子GOF包含的起始帧的编号，和/或该子GOF包含的点云的帧数可以直接或间接地推导出该子GOF包含的每帧点云在该子GOF所属的GOF中的位置。
在另一些实施例中：打包模块1201用于确定指示信息，该指示信息指示是否采用目标点云编码方法对点云组中的N个patch的辅助信息进行编码，目标点云编码方法包括上述图10所示的点云编码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。辅助信息编码模块1202用于将该指示信息编入码流。具体实现的过程中，编码器120还包括点云重构模块1203，用于根据该N帧点云中的多个patch的辅助信息，重构该N帧点云；该多个patch包括该N个patch。例如，结合图10，打包模块1201可以用于执行S301，辅助信息编码模块1202可以用于执行S302，点云重构模块1203可以用于执行S303。
作为一个示例，打包模块1201可以对应图5中的打包模块102，辅助信息编码模块1202可以对应图5中的辅助信息编码模块108。当然不限于此。点云重构模块1203具体可以对应图5中的点云重构模块112。需要说明的是，打包模块1201与重构模块1203可以直接连接，也可以是通过其他功能模块间接连接，例如，一种间接连接的示例可以参考图5中打包模块102与点云重构模块112之间的间接连接关系。
图13为本申请实施例提供的一种解码器130的示意性框图。解码器130可以用于执行图9或图11所示的点云解码方法。解码器130包括辅助信息解码模块1301和点云重构模块1302。
在一些实施例中，辅助信息解码模块1301用于解析码流，以得到语法元素，该语法元素包括点云组的并集占用图的辅助信息，该点云组包括N帧点云，N≥2，N是整数，该并集占用图是该N帧点云中具有匹配关系的N个patch的占用图的并集；根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息。点云重构模块1302用于基于该N帧点云中的多个patch的辅助信息，重构该N帧点云；其中，该多个patch包括该N个patch。例如，结合图9，辅助信息解码模块1301可以用于执行S201和S202，点云重构模块1302可以用于执行S203。
可选的，并集占用图的辅助信息包括以下信息中的至少一种：并集占用图在全局占用图中的坐标值、并集占用图在全局占用图中的尺寸信息和并集占用图对应在三维空间中的坐标值。全局占用图是对该点云组中的多个并集占用图进行打包得到的占用图。
可选的，上述语法元素还包括：该N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值。该情况下，在根据解析得到的该并集占用图的辅助信息，确定该N个patch的辅助信息的方面，辅助信息解码模块1301具体用于：根据解析得到的该并集占用图的辅助信息，以及该N个patch中的一个或多个patch的对应辅助信息与该并集占用图的辅助信息的差值，确定该N个patch的辅助信息。
可选的，该点云组是GOF或子GOF，该子GOF包括一个GOF中连续的多帧点云。
可选的，若该点云组是子GOF，上述语法元素还包括：表示该子GOF包含的点云(如每帧点云)在该子GOF所属的GOF中的位置的信息，例如，该子GOF包含的起始帧的编号，和/或该子GOF包含的点云的帧数。该情况下，辅助信息解码模块1301还用于，根据解析得到的表示该子GOF包含的点云在该子GOF所属的GOF中的位置的信息，确定该子GOF包含的点云(如每帧点云)在该子GOF所属的GOF中的位置。
在另一些实施例中：辅助信息解码模块1301用于解析码流，以得到指示信息，该指示信息指示是否采用目标点云解码方法对点云组中的N个patch的辅助信息进行解码，目标点云解码方法包括上述图11所示的点云解码方法；该点云组包括N帧点云，N≥2，N是整数；该N个patch是该N帧点云中具有匹配关系的N个patch。当该指示信息指示采用目标点云解码方法进行解码时，辅助信息解码模块1301和点云重构模块1302采用该目标点云解码方法对该N个patch的辅助信息进行处理。例如，结合图11，辅助信息解码模块1301可以用于执行S401，辅助信息解码模块1301和点云重构模块1302可以用于执行S402。其中，该处理过程中辅助信息解码模块1301和点云重构模块1302所执行的步骤可以参考上述实施例。例如，在该处理过程中，结合图9，辅助信息解码模块1301可以用于执行S201和S202，点云重构模块1302可以用于执行S203。
作为一个示例，辅助信息解码模块1301可以对应图7中的辅助信息解码模块204，点云重构模块1302可以对应图7中的点云重构模块205。
可以理解的，本申请实施例的编解码器中的各个模块为实现本申请对应的点云编解码方法中所包含的各种执行步骤的功能主体，即具备实现完整实现本申请对应的点云编解码方法中的各个步骤以及这些步骤的扩展及变形的功能主体，具体请参见本文中对图像滤波方法的介绍，为简洁起见，本文将不再赘述。
图14为用于本申请实施例的编码设备或解码设备(简称为译码设备160，或者点云译码设备160)的一种实现方式的示意性框图。其中，译码设备160可以包括处理器1610、存储器1630和总线系统1650。其中，处理器1610和存储器1630通过总线系统1650相连，该存储器1630用于存储指令，该处理器1610用于执行该存储器1630存储的指令，以执行本申请描述的各种点云编码或解码方法，尤其是基于当前图像块的块尺寸对当前图像块进行滤波的方法。为避免重复，这里不再详细描述。
在本申请实施例中，处理器1610可以是中央处理单元(central processing unit，CPU)，处理器1610还可以是其他通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器1630可以包括ROM设备或者RAM设备。任何其他适宜类型的存储设备也可以用作存储器1630。存储器1630可以包括由处理器1610使用总线系统1650访问的代码和数据1631。存储器1630可以进一步包括操作系统1633和应用程序1635，该应用程序1635包括允许处理器1610执行本申请描述的点云编码或解码方法(尤其是本申请描述的基于当前图像块的块尺寸对当前图像块进行滤波的方法)的至少一个程序。例如，应用程序1635可以包括应用1至N，其进一步包括执行在本申请描述的点云编码或解码方法的点云编码或解码应用(简称点云译码应用)。
总线系统1650除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线系统1650。
可选的，译码设备160还可以包括一个或多个输出设备，诸如显示器1670。在一个示例中，显示器1670可以是触感显示器，其将显示器与可操作地感测触摸输入的触感单元合并。显示器1670可以经由总线系统1650连接到处理器1610。
本领域技术人员能够领会，结合本文公开描述的各种说明性逻辑框、模块和算法步骤所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件来实施，那么各种说明性逻辑框、模块、和步骤描述的功能可作为一或多个指令或代码在计算机可读媒体上存储或传输，且由基于硬件的处理单元执行。计算机可读媒体可包含计算机可读存储媒体，其对应于有形媒体，例如数据存储媒体，或包括任何促进将计算机程序从一处传送到另一处的媒体(例如，根据通信协议)的通信媒体。以此方式，计算机可读媒体大体上可对应于(1)非暂时性的有形计算机可读存储媒体，或(2)通信媒体，例如信号或载波。数据存储媒体可为可由一或多个计算机或一或多个处理器存取以检索用于实施本申请中描述的技术的指令、代码和/或数据结构的任何可用媒体。计算机程序产品可包含计算机可读媒体。
作为实例而非限制，此类计算机可读存储媒体可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来存储指令或数据结构的形式的所要程序代码并且可由计算机存取的任何其它媒体。 并且，任何连接被恰当地称作计算机可读媒体。举例来说，如果使用同轴缆线、光纤缆线、双绞线、数字订户线(DSL)或例如红外线、无线电和微波等无线技术从网站、服务器或其它远程源传输指令，那么同轴缆线、光纤缆线、双绞线、DSL或例如红外线、无线电和微波等无线技术包含在媒体的定义中。但是，应理解，所述计算机可读存储媒体和数据存储媒体并不包括连接、载波、信号或其它暂时媒体，而是实际上针对于非暂时性有形存储媒体。如本文中所使用，磁盘和光盘包含压缩光盘(CD)、激光光盘、光学光盘、DVD和蓝光光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光以光学方式再现数据。以上各项的组合也应包含在计算机可读媒体的范围内。
可通过例如一或多个数字信号处理器(DSP)、通用微处理器、专用集成电路(ASIC)、现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指前述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文中所描述的各种说明性逻辑框、模块、和步骤所描述的功能可以提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或者并入在组合编解码器中。而且，所述技术可完全实施于一或多个电路或逻辑元件中。在一种示例下，编码器100及解码器200中的各种说明性逻辑框、单元、模块可以理解为对应的电路器件或逻辑元件。
本申请的技术可在各种各样的装置或设备中实施，包含无线手持机、集成电路(IC)或一组IC(例如，芯片组)。本申请中描述各种组件、模块或单元是为了强调用于执行所揭示的技术的装置的功能方面，但未必需要由不同硬件单元实现。实际上，如上文所描述，各种单元可结合合适的软件和/或固件组合在编码解码器硬件单元中，或者通过互操作硬件单元(包含如上文所描述的一或多个处理器)来提供。
以上所述，仅为本申请示例性的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。

Claims (38)

  1. 一种点云编码方法，其特征在于，包括：
    获取点云组的并集占用图的辅助信息；其中，所述点云组包括N帧点云，N≥2，N是整数，所述并集占用图是所述N帧点云中具有匹配关系的N个点云块patch的占用图的并集；所述并集占用图的辅助信息用于确定所述N个patch的辅助信息；
    将语法元素编入码流，所述语法元素包括所述并集占用图的辅助信息。
  2. 根据权利要求1所述的点云编码方法，其特征在于，所述并集占用图的辅助信息包括以下信息中的至少一种：所述并集占用图在全局占用图中的坐标值、所述并集占用图在全局占用图中的尺寸信息和所述并集占用图对应在三维空间中的坐标值；
    其中，所述全局占用图是对所述点云组中的多个所述并集占用图进行打包得到的占用图。
  3. 根据权利要求1或2所述的点云编码方法，其特征在于，所述语法元素还包括：所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值。
  4. 根据权利要求1至3任一项所述的点云编码方法，其特征在于，所述获取所述点云组的并集占用图的辅助信息，包括：
    基于所述N个patch的辅助信息，获取所述并集占用图的辅助信息。
  5. 根据权利要求4所述的点云编码方法，其特征在于，所述基于所述N个patch的辅助信息，获取所述并集占用图的辅助信息，包括：
    当所述并集占用图的辅助信息包括所述并集占用图在全局占用图中的坐标值时，将所述N个patch的占用图在对应点云的占用图中的最大坐标值或最小坐标值，作为所述并集占用图在全局占用图中的坐标值；
    和/或，当所述并集占用图的辅助信息包括所述并集占用图在全局占用图中的尺寸信息时，将所述N个patch的占用图在对应点云的占用图中的最大尺寸的信息作为所述并集占用图在全局占用图中的尺寸信息；
    和/或，当所述并集占用图的辅助信息包括所述并集占用图对应在三维空间中的坐标值时，将所述N个patch在对应点云中的最大坐标值或最小坐标值作为所述并集占用图对应在三维空间中的坐标值。
  6. 根据权利要求4或5所述的点云编码方法，其特征在于，所述方法还包括：
    根据所述N帧点云中的多个patch的辅助信息，重构所述N帧点云；其中，所述多个patch包括所述N个patch。
  7. 根据权利要求1至6任一项所述的点云编码方法，其特征在于，所述点云组是帧组GOF或子GOF，所述子GOF包括一个GOF中连续的多帧点云。
  8. 根据权利要求7所述的点云编码方法，其特征在于，若所述点云组是所述子GOF，所述语法元素还包括：表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息。
  9. 根据权利要求8所述的点云编码方法，其特征在于，所述表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息，包括：所述子GOF包含的起始帧的编号，和/或所述子GOF包含的点云的帧数。
  10. 一种点云解码方法，其特征在于，包括：
    解析码流，以得到语法元素，所述语法元素包括点云组的并集占用图的辅助信息，所述点云组包括N帧点云，N≥2，N是整数，所述并集占用图是所述N帧点云中具有匹配关系的N个点云块patch的占用图的并集；
    根据解析得到的所述并集占用图的辅助信息，确定所述N个patch的辅助信息；
    基于所述N帧点云中的多个patch的辅助信息，重构所述N帧点云；其中，所述多个patch包括所述N个patch。
  11. 根据权利要求10所述的点云解码方法，其特征在于，所述并集占用图的辅助信息包括以下信息中的至少一种：所述并集占用图在全局占用图中的坐标值、所述并集占用图在全局占用图中的尺寸信息和所述并集占用图对应在三维空间中的坐标值；
    其中，所述全局占用图是对所述点云组中的多个所述并集占用图进行打包得到的占用图。
  12. 根据权利要求10或11所述的点云解码方法，其特征在于，所述语法元素还包括：所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值；所述根据解析得到的所述并集占用图的辅助信息，确定所述N个patch的辅助信息，包括：
    根据解析得到的所述并集占用图的辅助信息，以及所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值，确定所述N个patch的辅助信息。
  13. 根据权利要求10至12任一项所述的点云解码方法，其特征在于，所述点云组是帧组GOF或子GOF，所述子GOF包括一个GOF中连续的多帧点云。
  14. 根据权利要求13所述的点云解码方法，其特征在于，若所述点云组是所述子GOF，所述语法元素还包括：表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息；所述方法还包括：
    根据解析得到的所述表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息，确定所述子GOF包含的点云在所述子GOF所属的GOF中的位置。
  15. 根据权利要求14所述的点云解码方法，其特征在于，所述表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息，包括：所述子GOF包含的起始帧的编号和/或所述子GOF包含的点云的帧数。
  16. 一种点云编码方法，其特征在于，包括：
    确定指示信息，所述指示信息指示是否采用目标点云编码方法对点云组中的N个点云块patch的辅助信息进行编码，所述目标点云编码方法包括权利要求1至9任一项所述的点云编码方法；所述点云组包括N帧点云，N≥2，N是整数；所述N个patch是所述N帧点云中具有匹配关系的N个patch；
    将所述指示信息编入码流。
  17. 一种点云解码方法，其特征在于，包括：
    解析码流，以得到指示信息，所述指示信息指示是否采用目标点云解码方法对点云组中的N个点云块patch的辅助信息进行解码，所述目标点云解码方法包括权利要求10至15任一项所述的点云解码方法；所述点云组包括N帧点云，N≥2，N是整数； 所述N个patch是所述N帧点云中具有匹配关系的N个patch；
    当所述指示信息指示采用所述目标点云解码方法进行解码时，采用所述目标点云解码方法对所述N个patch的辅助信息进行处理。
  18. 一种编码器，其特征在于，包括：
    打包模块，用于获取点云组的并集占用图的辅助信息；其中，所述点云组包括N帧点云，N≥2，N是整数，所述并集占用图是所述N帧点云中具有匹配关系的N个点云块patch的占用图的并集；所述并集占用图的辅助信息用于确定所述N个patch的辅助信息；
    辅助信息编码模块，用于将语法元素编入码流，所述语法元素包括所述并集占用图的辅助信息。
  19. 根据权利要求18所述的编码器，其特征在于，所述并集占用图的辅助信息包括以下信息中的至少一种：所述并集占用图在全局占用图中的坐标值、所述并集占用图在全局占用图中的尺寸信息和所述并集占用图对应在三维空间中的坐标值；
    其中，所述全局占用图是对所述点云组中的多个所述并集占用图进行打包得到的占用图。
  20. 根据权利要求18或19所述的编码器，其特征在于，所述语法元素还包括：所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值。
  21. 根据权利要求18至20任一项所述的编码器，其特征在于，所述打包模块具体用于：基于所述N个patch的辅助信息，获取所述并集占用图的辅助信息。
  22. 根据权利要求21所述的编码器，其特征在于，所述打包模块具体用于：
    当所述并集占用图的辅助信息包括所述并集占用图在全局占用图中的坐标值时，将所述N个patch的占用图在对应点云的占用图中的最大坐标值或最小坐标值，作为所述并集占用图在全局占用图中的坐标值；
    和/或，当所述并集占用图的辅助信息包括所述并集占用图在全局占用图中的尺寸信息时，将所述N个patch的占用图在对应点云的占用图中的最大尺寸的信息作为所述并集占用图在全局占用图中的尺寸信息；
    和/或，当所述并集占用图的辅助信息包括所述并集占用图对应在三维空间中的坐标值时，将所述N个patch在对应点云中的最大坐标值或最小坐标值作为所述并集占用图对应在三维空间中的坐标值。
  23. 根据权利要求21或22所述的编码器，其特征在于，所述编码器还包括：
    点云重构模块，用于根据所述N帧点云中的多个patch的辅助信息，重构所述N帧点云；其中，所述多个patch包括所述N个patch。
  24. 根据权利要求18至23任一项所述的编码器，其特征在于，所述点云组是帧组GOF或子GOF，所述子GOF包括一个GOF中连续的多帧点云。
  25. 根据权利要求24所述的编码器，其特征在于，若所述点云组是所述子GOF，所述语法元素还包括：表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息。
  26. 根据权利要求25所述的编码器，其特征在于，所述表示所述子GOF包含的 点云在所述子GOF所属的GOF中的位置的信息，包括：所述子GOF包含的起始帧的编号和/或所述子GOF包含的点云的帧数。
  27. 一种解码器，其特征在于，包括：
    辅助信息解码模块，用于解析码流，以得到语法元素，所述语法元素包括点云组的并集占用图的辅助信息，所述点云组包括N帧点云，N≥2，N是整数，所述并集占用图是所述N帧点云中具有匹配关系的N个点云块patch的占用图的并集；根据解析得到的所述并集占用图的辅助信息，确定所述N个patch的辅助信息；
    点云重构模块，用于基于所述N帧点云中的多个patch的辅助信息，重构所述N帧点云；其中，所述多个patch包括所述N个patch。
  28. 根据权利要求27所述的解码器，其特征在于，所述并集占用图的辅助信息包括以下信息中的至少一种：所述并集占用图在全局占用图中的坐标值、所述并集占用图在全局占用图中的尺寸信息和所述并集占用图对应在三维空间中的坐标值；
    其中，所述全局占用图是对所述点云组中的多个所述并集占用图进行打包得到的占用图。
  29. 根据权利要求27或28所述的解码器，其特征在于，所述语法元素还包括：所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值；
    在所述根据解析得到的所述并集占用图的辅助信息，确定所述N个patch的辅助信息的方面，所述辅助信息解码模块具体用于：根据解析得到的所述并集占用图的辅助信息，以及所述N个patch中的一个或多个patch的对应辅助信息与所述并集占用图的辅助信息的差值，确定所述N个patch的辅助信息。
  30. 根据权利要求27至29任一项所述的解码器，其特征在于，所述点云组是帧组GOF或子GOF，所述子GOF包括一个GOF中连续的多帧点云。
  31. 根据权利要求30所述的解码器，其特征在于，若所述点云组是所述子GOF，所述语法元素还包括：表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息；
    所述辅助信息解码模块还用于，根据解析得到的所述表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息，确定所述子GOF包含的点云在所述子GOF所属的GOF中的位置。
  32. 根据权利要求31所述的解码器，其特征在于，所述表示所述子GOF包含的点云在所述子GOF所属的GOF中的位置的信息，包括：所述子GOF包含的起始帧的编号和/或所述子GOF包含的点云的帧数。
  33. 一种编码器，其特征在于，包括：
    打包模块，用于确定指示信息，所述指示信息指示是否采用目标点云编码方法对点云组中的N个点云块patch的辅助信息进行编码，所述目标点云编码方法包括权利要求1至9任一项所述的点云编码方法；所述点云组包括N帧点云，N≥2，N是整数；所述N个patch是所述N帧点云中具有匹配关系的N个patch；
    辅助信息编码模块，用于将所述指示信息编入码流。
  34. 一种解码器，其特征在于，包括：辅助信息解码模块和点云重构模块；
    所述辅助信息解码模块，用于解析码流，以得到指示信息，所述指示信息指示是否采用目标点云解码方法对点云组中的N个点云块patch的辅助信息进行解码，所述目标点云解码方法包括权利要求10至15任一项所述的点云解码方法；所述点云组包括N帧点云，N≥2，N是整数；所述N个patch是所述N帧点云中具有匹配关系的N个patch；
    当所述指示信息指示采用所述目标点云解码方法进行解码时，所述辅助信息解码模块和所述点云重构模块采用所述目标点云解码方法对所述N个patch的辅助信息进行处理。
  35. 一种编码装置，其特征在于，包括存储器和处理器；其中，所述存储器用于存储程序代码；所述处理器用于调用所述程序代码，以执行如权利要求1至9任一项或权利要求16所述的点云编码方法。
  36. 一种解码装置，其特征在于，包括存储器和处理器；其中，所述存储器用于存储程序代码；所述处理器用于调用所述程序代码，以执行如权利要求10至15任一项或权利要求17所述的点云解码方法。
  37. 一种计算机可读存储介质，其特征在于，包括程序代码，当所述程序代码在计算机上运行时，使得所述计算机执行如权利要求1至9任一项或权利要求16所述的点云编码方法。
  38. 一种计算机可读存储介质，其特征在于，包括程序代码，当所述程序代码在计算机上运行时，使得所述计算机执行如权利要求10至15任一项或权利要求17所述的点云解码方法。
PCT/CN2019/103293 2018-09-30 2019-08-29 点云编解码方法和编解码器 WO2020063246A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201980060952.3A CN112740700A (zh) 2018-09-30 2019-08-29 点云编解码方法和编解码器
EP19866951.7A EP3852373A4 (en) 2018-09-30 2019-08-29 POINT CLOUD ENCODING PROCESS, POINT CLOUD DECODING PROCESS, ENCODER, AND DECODER
US17/217,404 US11388442B2 (en) 2018-09-30 2021-03-30 Point cloud encoding method, point cloud decoding method, encoder, and decoder

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811166410.8 2018-09-30
CN201811166410.8A CN110971912B (zh) 2018-09-30 2018-09-30 点云编解码方法、编解码器、编解码装置和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/217,404 Continuation US11388442B2 (en) 2018-09-30 2021-03-30 Point cloud encoding method, point cloud decoding method, encoder, and decoder

Publications (1)

Publication Number Publication Date
WO2020063246A1 true WO2020063246A1 (zh) 2020-04-02

Family

ID=69953331

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/103293 WO2020063246A1 (zh) 2018-09-30 2019-08-29 点云编解码方法和编解码器

Country Status (4)

Country Link
US (1) US11388442B2 (zh)
EP (1) EP3852373A4 (zh)
CN (2) CN110971912B (zh)
WO (1) WO2020063246A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114598692A (zh) * 2020-12-01 2022-06-07 腾讯科技（深圳）有限公司 点云文件传输方法、应用方法、装置、设备及存储介质
CN115396645A (zh) * 2022-08-18 2022-11-25 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质
US11961265B2 (en) 2018-12-13 2024-04-16 Huawei Technologies Co., Ltd. Point cloud encoding and decoding method and apparatus

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110971912B (zh) * 2018-09-30 2022-04-29 华为技术有限公司 点云编解码方法、编解码器、编解码装置和存储介质
CN111506519B (zh) * 2020-04-22 2021-04-27 上海安路信息科技股份有限公司 为fpga码点分配sram单元的方法及系统
CN111432210B (zh) * 2020-04-30 2021-10-19 中山大学 一种基于填充的点云属性压缩方法
CN111680095B (zh) * 2020-06-10 2021-01-12 上海城市地理信息系统发展有限公司 一种处理点云数据的方法、装置和电子设备
US20230334014A1 (en) * 2022-04-13 2023-10-19 Oracle International Corporation Implementing graph search with in-structure metadata of a graph-organized file system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107426559A (zh) * 2016-05-24 2017-12-01 汤姆逊许可公司 用于沉浸式视频格式的方法、装置和流
CN107770540A (zh) * 2016-08-21 2018-03-06 上海天荷电子信息有限公司 融合多种具有不同参考关系的基元的数据压缩方法和装置
WO2018130491A1 (en) * 2017-01-13 2018-07-19 Thomson Licensing Method, apparatus and stream for immersive video format

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10694210B2 (en) * 2016-05-28 2020-06-23 Microsoft Technology Licensing, Llc Scalable point cloud compression with transform, and corresponding decompression
US11514613B2 (en) * 2017-03-16 2022-11-29 Samsung Electronics Co., Ltd. Point cloud and mesh compression using image/video codecs
EP3496388A1 (en) * 2017-12-05 2019-06-12 Thomson Licensing A method and apparatus for encoding a point cloud representing three-dimensional objects
CN108122254B (zh) * 2017-12-15 2021-06-22 中国科学院深圳先进技术研究院 基于结构光的三维图像重建方法、装置及存储介质
CN110049323B (zh) * 2018-01-17 2021-09-07 华为技术有限公司 编码方法、解码方法和装置
EP3515066A1 (en) * 2018-01-19 2019-07-24 Thomson Licensing A method and apparatus for encoding and decoding three-dimensional scenes in and from a data stream
EP3595319A1 (en) * 2018-07-12 2020-01-15 InterDigital VC Holdings, Inc. Methods and apparatus for volumetric video transport
CN110944187B (zh) * 2018-09-19 2022-05-10 华为技术有限公司 点云编码方法和编码器
CN110971912B (zh) * 2018-09-30 2022-04-29 华为技术有限公司 点云编解码方法、编解码器、编解码装置和存储介质
WO2021002657A1 (ko) * 2019-07-04 2021-01-07 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107426559A (zh) * 2016-05-24 2017-12-01 汤姆逊许可公司 用于沉浸式视频格式的方法、装置和流
CN107770540A (zh) * 2016-08-21 2018-03-06 上海天荷电子信息有限公司 融合多种具有不同参考关系的基元的数据压缩方法和装置
WO2018130491A1 (en) * 2017-01-13 2018-07-19 Thomson Licensing Method, apparatus and stream for immersive video format

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LANYI HE: "Best-Effort Projection Based Attribute Compression for 3D Point Cloud", IEEE XPLORE, 1 March 2018 (2018-03-01), XP008150537 *
See also references of EP3852373A4

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11961265B2 (en) 2018-12-13 2024-04-16 Huawei Technologies Co., Ltd. Point cloud encoding and decoding method and apparatus
CN114598692A (zh) * 2020-12-01 2022-06-07 腾讯科技（深圳）有限公司 点云文件传输方法、应用方法、装置、设备及存储介质
CN114598692B (zh) * 2020-12-01 2023-03-17 腾讯科技（深圳）有限公司 点云文件传输方法、应用方法、装置、设备及存储介质
CN115396645A (zh) * 2022-08-18 2022-11-25 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质
CN115396645B (zh) * 2022-08-18 2024-04-19 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
EP3852373A1 (en) 2021-07-21
CN110971912A (zh) 2020-04-07
US11388442B2 (en) 2022-07-12
US20210219000A1 (en) 2021-07-15
CN112740700A (zh) 2021-04-30
CN110971912B (zh) 2022-04-29
EP3852373A4 (en) 2021-11-03

Similar Documents

Publication Publication Date Title
WO2020063246A1 (zh) 点云编解码方法和编解码器
WO2020001565A1 (zh) 点云编解码方法和编解码器
US20210029381A1 (en) Method and apparatus for obtaining global matched patch
WO2020011265A1 (zh) 点云编解码方法和编解码器
WO2020151496A1 (zh) 点云的编解码方法及装置
US11875538B2 (en) Point cloud encoding method and encoder
WO2020063294A1 (zh) 点云编解码方法及编解码器
WO2020147379A1 (zh) 点云滤波方法、装置及存储介质
WO2020119487A1 (zh) 点云的编解码方法及装置
US20220007037A1 (en) Point cloud encoding method and apparatus, point cloud decoding method and apparatus, and storage medium
WO2020063718A1 (zh) 点云编解码方法和编解码器
JP2022513484A (ja) 点群符号化方法及びエンコーダ
WO2020143725A1 (zh) 点云译码方法及译码器
WO2020057338A1 (zh) 点云编码方法和编码器
WO2020015517A1 (en) Point cloud encoding method, point cloud decoding method, encoder and decoder
WO2020187283A1 (zh) 点云编码方法、点云解码方法、装置及存储介质
WO2020220941A1 (zh) 点云编码方法、点云解码方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19866951

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019866951

Country of ref document: EP

Effective date: 20210415