WO2020119509A1 - 点云编解码方法和编解码器 - Google Patents

点云编解码方法和编解码器 Download PDF

Info

Publication number
WO2020119509A1
WO2020119509A1 PCT/CN2019/122485 CN2019122485W WO2020119509A1 WO 2020119509 A1 WO2020119509 A1 WO 2020119509A1 CN 2019122485 W CN2019122485 W CN 2019122485W WO 2020119509 A1 WO2020119509 A1 WO 2020119509A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel block
processed
boundary pixel
target
boundary
Prior art date
Application number
PCT/CN2019/122485
Other languages
English (en)
French (fr)
Inventor
蔡康颖
张德军
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020119509A1 publication Critical patent/WO2020119509A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • the present application relates to the technical field of codecs, in particular to a point cloud codec method and codec.
  • 3d sensor such as 3d scanner
  • 3d scanner 3d scanner
  • the embodiments of the present application provide a point cloud codec method and a codec, which help to improve codec performance.
  • a point cloud decoding method which includes: determining whether an adjacent pixel block in the spatial domain of the boundary pixel block to be processed is an invalid pixel block based on the occupancy map of the point cloud to be decoded, and determining the Type; wherein, the invalid pixel block includes pixel blocks whose values of the contained pixels are not all 0 and do not belong to the target point cloud block patch occupancy map; or, the invalid pixel block includes pixel blocks whose values of the included pixels are all 0, And the pixel value of the contained pixels is not all 0 and does not belong to the pixel block of the target patch occupancy map; the target patch occupancy map is the patch occupancy map to which the boundary pixel block to be processed belongs; according to the type of the boundary pixel block to be processed, it is reconstructed to be translated Code point cloud.
  • the point cloud to be decoded is the current frame point cloud. Because the point cloud to be compressed is divided into multiple patches, and then the patch is mapped to the patch occupancy map, and the point cloud coding and decoding method for packaging each patch occupancy map will cause a large reconstruction error near the patch boundary. Therefore, in order to ensure the quality of the reconstructed point cloud, it is usually necessary to perform special processing on the boundary of the patch occupancy graph, such as filtering and hole filling.
  • the type of the boundary pixel block to be processed is determined based on whether the spatial adjacent boundary pixel block of the boundary pixel block to be processed is an invalid pixel block, so that the point cloud is reconstructed based on the type.
  • the invalid pixel block includes pixel blocks whose values of the included pixels are not all 0 and do not belong to the target patch occupancy map. In this way, it is helpful to identify the boundary pixel blocks with large reconstruction errors in the occupancy map of the point cloud to be decoded, so that different processing methods can be adopted to ensure the quality of the reconstructed point cloud, so the system can be improved Codec performance.
  • the method further includes: when the number of invalid pixel blocks in the adjacent pixel blocks of the current boundary pixel block of the occupancy map of the point cloud to be decoded is greater than or equal to a preset threshold, determining The current boundary pixel block is a boundary pixel block to be processed.
  • the spatial neighboring pixel blocks of the preset orientation of the boundary pixel block to be processed are invalid pixel blocks, it is determined that the invalid pixels in the boundary pixel block to be processed are preset in the boundary pixel block to be processed Orientation; wherein, the preset orientation is one or a combination of at least two of right above, right below, right left, right right, top left, top right, bottom left, and bottom right.
  • reconstructing the point cloud to be decoded according to the type of the boundary pixel block to be processed includes: performing a target operation on the boundary pixel block to be processed according to the type of the boundary pixel block to be processed , Get the pixel block that has been processed by the target; and reconstruct the point cloud to be decoded according to the processed occupancy map.
  • the processed occupancy map includes the pixel block that has been processed by the target; where, according to the type of the border pixel block to be processed , Adopt the corresponding target processing method to perform the target operation on the boundary pixel block to be processed to obtain the pixel block subjected to the target operation, including: when the target operation is a zero-setting operation, the corresponding target processing method is adopted according to the type of the boundary pixel block to be processed Set the value of the pixel at the target position of the boundary pixel block to be processed to 0 to obtain a pixel block that is set to 0.
  • the target position corresponding to different processing methods is different; or, when the target operation is to set the operation, according to the boundary pixel block to be processed Type, the corresponding target processing method is used to set the value of the pixel at the target position of the boundary pixel block to be processed to 1 to obtain the set pixel block, the target position corresponding to different processing methods is different; or, when the target operation is an expansion operation At this time, according to the type of the boundary pixel block to be processed, the corresponding target processing method is used to perform the expansion operation on the boundary pixel block to be processed to obtain the pixel block after the expansion operation, and the radius of the convolution kernel in different processing methods is different.
  • the zero-setting operation described in the embodiments of the present application is an operation for making pixels in one pixel block unoccupied
  • the one-setting operation is an operation for making pixels in one pixel block unoccupied.
  • the execution result of the zero-setting operation is that the value of a pixel is 0, and the execution result of the zero-setting operation is that the value of a pixel is 1, which is also used as an example in the following description.
  • 0 and 1 are just to distinguish between possession and non-possession, and can be replaced with other values or a range in actual implementation. For example, you can use "4" for possession, non-4 for non-occupancy, and so on.
  • the target pixel block to be processed is subjected to a target operation using a corresponding target processing method to obtain the pixel block subjected to the target operation, including: according to the multiple types of the boundary pixel block Mapping relationship with multiple processing methods to determine the processing method corresponding to the type of boundary pixel block to be processed; if the type of boundary pixel block to be processed corresponds to one processing method, the processing corresponding to the type of boundary pixel block to be processed As the target processing method; or, if the type of boundary pixel block to be processed corresponds to multiple processing methods, one of the multiple processing methods corresponding to the type of boundary pixel block to be processed is used as the target processing method; the target is used
  • the processing method performs a target operation on the boundary pixel block to be processed to obtain a pixel block subjected to the target operation.
  • a corresponding target processing method is used to perform a target operation on the boundary pixel block to be processed to obtain a pixel block subjected to the target operation, including: according to the type of the boundary pixel block to be processed Look up the table to get the processing method corresponding to the type of boundary pixel block to be processed.
  • the table includes the mapping relationship between the types of boundary pixel blocks and multiple processing methods; if the type of boundary pixel block to be processed corresponds to a processing method, The processing method corresponding to the type of the boundary pixel block to be processed is taken as the target processing method; or, if the type of the boundary pixel block to be processed corresponds to multiple processing methods, the multiple processing methods corresponding to the type of the boundary pixel block to be processed One of the processing methods is used as the target processing method; the target processing method is used to perform the target operation on the boundary pixel block to be processed to obtain the pixel block subjected to the target operation.
  • one of the multiple processing methods corresponding to the type of boundary pixel block to be processed is used as the target processing method, including: according to whether the adjacent pixel block of the target airspace of the boundary pixel block to be processed is Empty invalid pixel block, select one processing method from the various processing methods corresponding to the type of boundary pixel block to be processed as the target processing method; the adjacent pixel block of the target airspace is the spatial phase of the invalid pixel block to be processed boundary pixel block Adjacent pixel blocks.
  • one processing method is selected as the target from a variety of processing methods corresponding to the type of the boundary pixel block to be processed Processing methods, including: if the target airspace adjacent pixel block is an empty invalid pixel block, then select the first processing method as the target processing method from the multiple processing methods corresponding to the type of boundary pixel block to be processed; if the target airspace adjacent pixel If the block is a non-empty invalid pixel block, then the second processing method is selected as the target processing method from a variety of processing methods corresponding to the type of boundary pixel block to be processed; where, when the target operation is a zero-setting operation, the first processing method corresponds to The range of the target position is greater than the range of the target position corresponding to the second processing mode; or, when the target operation is the set-one operation, the range of the target position corresponding to the first processing mode is smaller than the range of the target position
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: adjacent to the boundary pixel block to be processed and located directly above, directly below, to the left, and to the right of the boundary pixel block to be processed Pixel block; if the adjacent pixel block in the airspace in the preset direction of the boundary pixel block to be processed is an invalid pixel block, and all other adjacent pixel blocks in the airspace are valid pixel blocks, the orientation information is: invalid in the boundary pixel block to be processed
  • the pixel is located in a preset direction in the boundary pixel block to be processed; the preset direction includes one or a combination of at least two of the top, bottom, left, and right directions; or, if the The pixel blocks directly above and to the right are invalid pixel blocks, and the pixel blocks directly below and to the left of the boundary pixel block to be processed are valid pixel blocks, then the orientation information is: the invalid pixels in the boundary pixel block to be processed are located to be processed The upper right of the boundary pixel block; or,
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include pixels adjacent to the boundary pixel block to be processed and located at the upper left, upper right, lower left, and lower right of the boundary pixel block to be processed Block; if the adjacent pixel block in the airspace in the preset direction of the boundary pixel block to be processed is an invalid pixel block, and the other adjacent pixel blocks in the airspace are all valid pixel blocks, the orientation information is: invalid pixels in the boundary pixel block to be processed A preset direction located in the boundary pixel block to be processed; the preset direction includes one or at least two of upper left, upper right, lower left, and lower right.
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: adjacent to the boundary pixel block to be processed and located directly above, directly below, to the left, or right of the boundary pixel block to be processed Square, upper left, upper right, lower left, and lower right pixel blocks; if the neighboring pixel blocks in the airspace in the preset direction of the boundary pixel block to be processed are invalid pixel blocks, and the other adjacent pixel blocks in the airspace are all valid pixel blocks ,
  • the orientation information is: the invalid pixels in the boundary pixel block to be processed are located in the preset direction in the boundary pixel block to be processed; the preset direction includes upper left, upper right, lower left, or lower right.
  • the target position is in the boundary pixel block to be processed, and the position of the invalid pixel whose distance from the target effective pixel is greater than or equal to the preset threshold is located; or , The target position is in the boundary pixel block to be processed, and the distance between the effective pixel and the straight line where the target effective pixel is greater than or equal to the preset threshold is located; the straight line is related to the type of the boundary pixel block to be processed;
  • the target position is in the boundary pixel block to be processed, and the invalid pixel whose distance from the target effective pixel is less than or equal to the preset threshold is located; or , The target position is in the boundary pixel block to be processed, and the position of the invalid pixel whose distance from the straight line where the target valid pixel is located is less than or equal to the preset threshold; the straight line is related to the type of the boundary pixel block to be processed.
  • the occupancy map of the point cloud to be decoded includes a first to-be-processed boundary pixel block and a second to-be-processed boundary pixel block;
  • the type of the first to-be-processed boundary pixel block is the first type, the second to-be-processed
  • the type of the processed boundary pixel block is the second type;
  • the first type and the second type correspond to the same orientation information;
  • the invalid pixel block of the preset orientation in the adjacent pixel block in the air domain of the first boundary pixel block to be processed is null and invalid
  • the invalid pixel block with the preset orientation in the adjacent pixel block in the spatial domain of the second to-be-processed boundary pixel block is a non-empty invalid pixel block.
  • the point cloud to be decoded is a point cloud to be encoded. If the type of the boundary pixel block to be processed corresponds to multiple processing methods; the method further includes: coding the identification information into the code stream, and the identification information indicates that the processing is to be processed The target processing method of the boundary pixel block.
  • the point cloud to be decoded is a point cloud to be decoded.
  • the target processing method is used to perform the target operation on the boundary pixel block to be processed to obtain the target operation
  • the pixel block includes: parsing the code stream according to the type of the boundary pixel block to be processed to obtain identification information; the identification information indicates the target processing method; adopting the target processing method indicated by the identification information to perform the target operation on the boundary pixel block to be processed to obtain The pixel block operated by the target.
  • the point cloud to be decoded is a point cloud to be decoded
  • the method further includes: parsing the code stream to obtain the size information of the boundary pixel block to be processed of the point cloud to be decoded; and treating the decoded point according to the size information
  • the cloud occupancy map is divided to obtain one or more boundary pixel blocks to be processed.
  • a point cloud coding method which includes: determining instruction information used to indicate whether to process the occupancy map of the point cloud to be coded according to the target coding method; the target coding method includes the first aspect or the first
  • any possible design provides a point cloud decoding method (specifically, a point cloud coding method); the indication information is encoded into the code stream.
  • a point cloud decoding method which includes: parsing a code stream to obtain indication information used to indicate whether to process an occupancy map of a point cloud to be decoded according to a target decoding method; the target decoding method includes the first A point cloud decoding method (specifically a point cloud decoding method) provided by any one of the possible designs of the first aspect or the first aspect; when the indication information is used to instruct the occupancy map of the point cloud to be decoded according to the target decoding method , Process the occupancy map of the decoded point cloud according to the target decoding method.
  • a decoder including: a classification module that determines whether the boundary to be processed is based on whether an adjacent pixel block in the spatial domain of the boundary pixel block to be processed of the occupancy map of the point cloud to be decoded is an invalid pixel block The type of pixel block; wherein, the invalid pixel block includes a pixel block whose pixel value is not all 0 and does not belong to the target point cloud block patch occupancy map; or, the invalid pixel block includes the value of the included pixel Pixel blocks that are all 0, and pixel blocks that contain pixels whose values are not all 0 and do not belong to the target patch occupancy map; the target patch occupancy map is the patch occupancy map to which the boundary pixel block to be processed belongs, reconstruct The module reconstructs the point cloud to be decoded according to the type of the boundary pixel block to be processed.
  • the classification module may correspond to the classification submodule in the occupancy graph filtering module 112 in FIG. 2.
  • the reconstruction module may correspond to the preprocessing submodule in the
  • an encoder including: an auxiliary information encoding module, configured to determine indication information and encode the indication information into a code stream; the indication information is used to indicate whether to encode the point cloud according to the target encoding method
  • the occupancy graph is processed; the target encoding method includes the point cloud decoding method (specifically, the point cloud encoding method) provided by the first aspect or any possible design of the first aspect.
  • a decoder including: an auxiliary information decoding module for parsing a code stream to obtain indication information, where the indication information is used to indicate whether to process an occupancy map of a point cloud to be decoded according to a target decoding method;
  • the target decoding method includes the point cloud decoding method (specifically, the point cloud decoding method) provided by the first aspect or any possible design of the first aspect.
  • the occupancy graph filtering module is configured to process the occupancy graph of the decoded point cloud according to the target decoding method when the indication information is used to instruct the occupancy graph of the point cloud to be decoded according to the target decoding method.
  • a decoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to perform the first aspect or any of the first aspect
  • a possible design provides a point cloud decoding method.
  • an encoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to perform the point cloud encoding method provided in the second aspect.
  • a decoding device including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to perform the point cloud encoding method provided in the third aspect.
  • the present application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to perform any point cloud decoding method as provided in the first aspect and its possible design described above.
  • the present application also provides a computer-readable storage medium, including program code, which, when run on a computer, causes the computer to execute the point cloud coding method provided in the second aspect.
  • the present application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to execute the point cloud coding method provided in the third aspect.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system that can be used in an example of an embodiment of the present application
  • FIG. 2 is a schematic block diagram of an example encoder that can be used in an embodiment of the present application
  • FIG. 3 is a schematic diagram of a point cloud, a patch of a point cloud, and an occupancy diagram of a point cloud applicable to embodiments of the present application;
  • FIG. 4 is a schematic block diagram of a decoder that can be used in an example of an embodiment of the present application
  • FIG. 5 is a schematic flowchart of a point cloud decoding method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a target location provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another target location provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of another target location provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of the index, discrimination method diagram, schematic diagram, and description information of the pixel block type provided by the embodiment of the present application.
  • FIG. 10 is a schematic diagram of a pixel for determining a target position provided by an embodiment of this application.
  • FIG. 11 is a schematic diagram of another pixel for determining a target position provided by an embodiment of this application.
  • FIG. 12 is a schematic diagram of another pixel for determining a target position provided by an embodiment of the present application.
  • FIG. 13 is a schematic diagram of another pixel for determining a target position provided by an embodiment of this application.
  • FIG. 14 is a schematic diagram of another pixel for determining a target position provided by an embodiment of this application.
  • 15 is a schematic diagram of another pixel for determining a target position provided by an embodiment of this application.
  • 16 is a schematic diagram of another pixel for determining a target position provided by an embodiment of this application.
  • 17 is a schematic diagram of several convolution kernels applicable to an embodiment of the present application.
  • FIG. 18 is a schematic diagram of pixel blocks before filling two types of boundary pixel blocks to be processed of type 1 provided by an embodiment of the present application;
  • 19 is a schematic diagram of a code stream structure provided by an embodiment of this application.
  • FIG. 20 is a schematic flowchart of a point cloud coding method provided by an embodiment of the present application.
  • 21 is a schematic flowchart of a point cloud decoding method provided by an embodiment of the present application.
  • 22 is a schematic block diagram of a decoder provided by an embodiment of this application.
  • 23 is a schematic block diagram of another decoder provided by an embodiment of the present application.
  • 24 is a schematic block diagram of another decoder provided by an embodiment of the present application.
  • 25 is a schematic block diagram of an encoder provided by an embodiment of this application.
  • 26 is a schematic block diagram of a decoder provided by an embodiment of this application.
  • FIG. 27 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.
  • At least one (species) in the embodiments of the present application includes one (species) or a plurality (species).
  • Multiple (species) means two (species) or more than two (species).
  • at least one of A, B, and C includes the presence of A alone, B alone, A and B, A and C, B and C, and A, B, and C.
  • “/" means “or”, for example, A/B can mean A or B;
  • and/or” in this text is merely an association relationship describing an associated object, It means that there can be three kinds of relationships, for example, A and/or B.
  • FIG. 1 is a schematic block diagram of a point cloud decoding system 1 that can be used in an example of an embodiment of the present application.
  • the term "point cloud coding" or “coding” may generally refer to point cloud coding or point cloud decoding.
  • the encoder 100 of the point cloud decoding system 1 may encode the point cloud to be encoded according to any point cloud encoding method proposed in this application.
  • the decoder 200 of the point cloud decoding system 1 may decode the point cloud to be decoded according to the point cloud decoding method corresponding to the point cloud encoding method used by the encoder proposed in this application.
  • the point cloud decoding system 1 includes a source device 10 and a destination device 20.
  • the source device 10 generates encoded point cloud data. Therefore, the source device 10 may be referred to as a point cloud encoding device.
  • the destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may be referred to as a point cloud decoding device.
  • Various implementations of source device 10, destination device 20, or both may include one or more processors and memory coupled to the one or more processors.
  • the memory may include, but is not limited to, random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), electrically erasable programmable read-only memory (electrically erasable, programmable-read-only memory, EEPROM) ), flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures accessible by the computer, as described herein.
  • random access memory random access memory
  • RAM read-only memory
  • read-only memory read-only memory
  • ROM read-only memory
  • electrically erasable programmable read-only memory electrically erasable, programmable-read-only memory, EEPROM
  • flash memory or any other medium that can be used to store the desired program code in the form of instructions or data structures accessible by the computer, as described herein.
  • Source device 10 and destination device 20 may include various devices, including desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart” phones, etc. Devices, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.
  • Link 30 may include one or more media or devices capable of moving the encoded point cloud data from source device 10 to destination device 20.
  • link 30 may include one or more communication media that enable source device 10 to send encoded point cloud data directly to destination device 20 in real time.
  • the source device 10 may modulate the encoded point cloud data according to a communication standard (eg, a wireless communication protocol), and may send the modulated point cloud data to the destination device 20.
  • the one or more communication media may include wireless and/or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (eg, the Internet).
  • the one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20.
  • the encoded data may be output from the output interface 140 to the storage device 40.
  • the encoded point cloud data can be accessed from the storage device 40 through the input interface 240.
  • the storage device 40 may include any of a variety of distributed or locally accessed data storage media, such as a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc (read-only disc-read) only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded point cloud data.
  • the storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded point cloud data generated by the source device 10.
  • the destination device 20 may access the stored point cloud data from the storage device 40 via streaming or download.
  • the file server may be any type of server capable of storing encoded point cloud data and transmitting the encoded point cloud data to the destination device 20.
  • Example file servers include network servers (for example, for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives.
  • the destination device 20 can access the encoded point cloud data through any standard data connection, including an Internet connection.
  • This may include a wireless channel (eg, Wi-Fi connection), a wired connection (eg, digital subscriber line (DSL), cable modem, etc.), or a coded point cloud suitable for accessing storage on a file server A combination of both of the data.
  • the transmission of the encoded point cloud data from the storage device 40 may be streaming transmission, download transmission, or a combination of both.
  • the point cloud decoding system 1 illustrated in FIG. 1 is only an example, and the technology of the present application can be applied to point cloud decoding (eg, point cloud decoding) that does not necessarily include any data communication between the point cloud encoding device and the point cloud decoding device Cloud coding or point cloud decoding) device.
  • data is retrieved from local storage, streamed on the network, and so on.
  • the point cloud encoding device may encode the data and store the data to the memory, and/or the point cloud decoding device may retrieve the data from the memory and decode the data.
  • encoding and decoding are performed by devices that do not communicate with each other but only encode data to and/or retrieve data from memory and decode the data.
  • the source device 10 includes a data source 120, an encoder 100 and an output interface 140.
  • the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter (or referred to as a transmitter).
  • the data source 120 may include a point cloud capture device (eg, a camera), a point cloud archive containing previously captured point cloud data, a point cloud feed interface to receive point cloud data from a point cloud content provider, and/or For computer graphics systems that generate point cloud data, or a combination of these sources of point cloud data.
  • the encoder 100 may encode point cloud data from the data source 120.
  • the source device 10 sends the encoded point cloud data directly to the destination device 20 via the output interface 140.
  • the encoded point cloud data may also be stored on storage device 40 for later access by destination device 20 for decoding and/or playback.
  • the destination device 20 includes an input interface 240, a decoder 200 and a display device 220.
  • input interface 240 includes a receiver and/or a modem.
  • the input interface 240 may receive the encoded point cloud data via the link 30 and/or from the storage device 40.
  • the display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. In general, the display device 220 displays decoded point cloud data.
  • the display device 220 may include various display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-multiplexer- demultiplexer (MUX-DEMUX) unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams.
  • MUX-DEMUX multiplexer-multiplexer- demultiplexer
  • the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as user datagram protocol (UDP).
  • the encoder 100 and the decoder 200 can each be implemented as any of a variety of circuits such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (applications) specific integrated circuit (ASIC), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof.
  • DSPs digital signal processors
  • ASIC application specific integrated circuits
  • FPGA field-programmable gate array
  • the device may store the instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware
  • Any of the foregoing including hardware, software, a combination of hardware and software, etc.
  • Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, any of which may be integrated as a combined encoder/decoder in the corresponding device Part of the decoder (codec).
  • This application may generally refer to the encoder 100 as another device that “signals” or “sends” certain information to, for example, the decoder 200.
  • the terms “signaling” or “sending” may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transfer can occur in real time or almost real time. Alternatively, this communication may occur after a period of time, for example, may occur when the syntax element is stored in the encoded bitstream to a computer-readable storage medium at the time of encoding, and the decoding device may then store the syntax element to this medium At any time.
  • FIG. 2 it is a schematic block diagram of an encoder 100 that can be used in an example of an embodiment of the present application.
  • Fig. 2 takes the MPEG (Moving Picture Expert Group) Point Cloud Compression (PCC) coding framework as an example for illustration.
  • the encoder 100 may include a patch information generation module 101, a packaging module 102, a depth map generation module 103, a texture map generation module 104, a first filling module 105, an image or video-based encoding module 106, an occupation The graph encoding module 107, the auxiliary information encoding module 108, the multiplexing module 109, and the like.
  • the encoder 100 may further include a point cloud filtering module 110, a point cloud reconstruction module 111, an occupancy map filtering module 112, and the like. among them:
  • the patch information generation module 101 is used to divide a frame of point cloud into a plurality of patches by a certain method, and obtain relevant information of the generated patch.
  • patch refers to a set of points in a frame of point cloud, usually a connected area corresponds to a patch.
  • the relevant information of the patch may include but not limited to at least one of the following information: the number of patches divided by the point cloud, the position information of each patch in three-dimensional space, the index of the normal coordinate axis of each patch, each The depth map generated by projecting a patch from three-dimensional space to two-dimensional space, the size of the depth map of each patch (such as the width and height of the depth map), and the occupancy map generated by projecting each patch from three-dimensional space to two-dimensional space.
  • the part of the relevant information such as the number of patches divided by the point cloud, the index of the normal coordinate axis of each patch, the size of the depth map of each patch, the location information of each patch in the point cloud, each The size information of the patch's occupancy map, etc., may be sent as auxiliary information to the auxiliary information encoding module 108 for encoding (ie, compression encoding).
  • the occupancy map of each patch can be sent to the packaging module 102 for packaging.
  • the patches of the point cloud are arranged in a specific order, for example, according to the width/height descending (or ascending) order of the occupancy map of each patch; Then, according to the order of the arranged patches, the patch occupancy map is inserted into the available area of the point cloud occupancy map in sequence to obtain the point cloud occupancy map.
  • the specific location information of each patch in the point cloud occupancy map and the depth map of each patch, etc. can be sent to the depth map generation module 103.
  • the packaging module 102 can send the occupancy map of the point cloud to the occupancy map encoding module 107 for encoding.
  • the occupancy map of the point cloud can be used to guide the depth map generation module 103 to generate the depth map of the point cloud and the guidance texture map generation module 104 to generate the texture map of the point cloud.
  • FIG. 3 it is a schematic diagram of a point cloud, a patch of a point cloud, and an occupancy map of a point cloud applicable to the embodiments of the present application.
  • (a) in FIG. 3 is a schematic diagram of a frame of point cloud
  • (b) in FIG. 3 is a schematic diagram of a patch of a point cloud obtained based on (a) in FIG. 3
  • ( c) The figure is a schematic diagram of the occupancy map of the point cloud obtained by packaging the occupancy map of each patch obtained by mapping each patch shown in (b) in FIG. 3 onto a two-dimensional plane.
  • the depth map generation module 103 is used to generate a depth map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud and depth information, and send the generated depth map to the first filling module 105. Fill the blank pixels in the depth map to obtain a filled depth map.
  • the texture map generation module 104 is used to generate a texture map of the point cloud according to the occupancy map of the point cloud, the occupancy map of each patch of the point cloud and texture information, and send the generated texture map to the first filling module 105. Fill the blank pixels in the texture image to obtain a filled texture image.
  • the filled depth map and the filled texture map are sent by the first filling module 105 to the image or video-based encoding module 106 to perform image or video-based encoding.
  • the image or video-based encoding module 106, occupancy map encoding module 107, and auxiliary information encoding module 108 send the resulting encoding result (ie, code stream) to the multiplexing module 109 to merge into one code stream.
  • the code stream may be sent to the output interface 140.
  • the encoding result (ie, code stream) obtained by the image or video encoding module 106 is sent to the point cloud reconstruction module 111 for point cloud reconstruction to obtain a reconstructed point cloud (specifically, the reconstructed point Cloud geometry information).
  • a reconstructed point cloud specifically, the reconstructed point Cloud geometry information.
  • perform video decoding on the encoded depth map obtained by the image or video encoding module 106 to obtain the decoded depth map of the point cloud, and use the decoded depth map, the occupancy map of the point cloud, and auxiliary information of each patch To obtain the geometric information of the reconstructed point cloud.
  • the geometric information of the point cloud refers to the coordinate values of the points in the point cloud (for example, each point in the point cloud) in three-dimensional space.
  • the “occupation map of the point cloud” herein may be an occupancy map obtained after the point cloud is filtered (or referred to as smoothing processing) by the occupancy map filtering module 112.
  • the point cloud reconstruction module 111 may also send the texture information of the point cloud and the reconstructed point cloud geometry information to the coloring module, the coloring module is used to color the reconstructed point cloud to obtain the reconstructed point cloud Texture information.
  • the texture map generation module 104 may also generate a texture map of the point cloud based on information obtained by filtering the reconstructed point cloud geometric information via the point cloud filter module 110.
  • the occupancy graph filtering module 112 may be located in the packaging module 102 and the point cloud reconstruction module 111, and is used to filter the occupancy graph of the filled point cloud.
  • the encoder 100 further includes a second filling unit for filling the occupancy map of the point cloud generated by the occupancy map filtering module 112.
  • the occupancy graph filtering module 112 performs filling of the point cloud occupancy graph from the packaging module 102.
  • the occupancy map filtering module 112 may include a classification submodule, a preprocessing submodule, and a filtering submodule.
  • a filling submodule may also be included for filling the occupancy map of the point cloud generated by the occupancy map filtering module 112.
  • the classification submodule is used to classify the boundary pixel blocks of the occupancy map of the current frame point cloud
  • the preprocessing submodule is used to process the occupancy map of the point cloud of the current frame based on the classification result (such as the operation of setting 0 or setting 1) Or dilation operation, etc.)
  • a filtering sub-module which is used to filter the occupied map of the processed point cloud (also called smoothing).
  • the occupancy graph filtering module 112 is also connected to the packaging module 102 and the auxiliary information encoding module 108, as shown by the dotted line in FIG. 2.
  • the occupancy map filtering module 112 is also used to determine the target processing mode corresponding to the boundary pixel block to be processed according to the occupancy map of the point cloud sent by the packaging module 102, and send the identification information of the target processing mode as auxiliary information to the auxiliary information encoding module 108, And the auxiliary information encoding module 108 encodes the identification information into the code stream.
  • the identification information of the target processing method is used as auxiliary information and the auxiliary information encoding module 108 encodes it into the code stream as an example for illustration.
  • the identification of the target processing method The information can also be encoded into a code stream by an encoding module independent of the auxiliary information encoding module 108, and the code stream is sent to the multiplexing module 109 to obtain a combined code stream.
  • the occupancy map filtering module 112 determines the target processing mode corresponding to the boundary pixel block to be processed according to the occupancy map of the point cloud sent by the packaging module 102 as an example.
  • the filtering module 112 can also determine the target processing method without depending on the occupancy map of the point cloud sent by the packaging module 102. In this case, the occupancy graph filtering module 112 may not be connected to the packaging module 102.
  • the occupancy graph filtering module 112 may also be connected to the patch information generation module 101, and used to obtain auxiliary information such as Block2PatchIndex from the patch information generation module 101.
  • the encoder 100 shown in FIG. 2 is only an example, and in specific implementation, the encoder 100 may include more or fewer modules than those shown in FIG. 2. This embodiment of the present application does not limit this.
  • FIG. 4 it is a schematic block diagram of a decoder 200 that can be used in an example of an embodiment of the present application.
  • the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy graph decoding module 203, an auxiliary information decoding module 204, a point cloud reconstruction module 205, and a point cloud filtering module 206 and the texture information reconstruction module 207 of the point cloud.
  • the decoder 200 may include an occupancy map filtering module 208. among them:
  • the demultiplexing module 201 is used to send the input code stream (ie, the combined code stream) to the corresponding decoding module. Specifically, the code stream containing the encoded texture map and the coded depth map are sent to the image or video-based decoding module 202; the code stream containing the encoded occupancy map is sent to the occupancy map decoding module 203 , The code stream containing the encoded auxiliary information is sent to the auxiliary information decoding module 204.
  • the occupancy graph decoding module 203 is configured to decode the received code stream containing the encoded occupancy graph, and send the decoded occupancy graph information to the point cloud reconstruction module 205.
  • the occupancy map information sent to the point cloud reconstruction module 205 may be information of the occupancy map obtained after filtering by the occupancy map filtering module 208.
  • the auxiliary information decoding module 204 is configured to decode the received encoded auxiliary information, and send the decoded information indicating the auxiliary information to the point cloud reconstruction module 205.
  • the point cloud reconstruction module 205 is used to reconstruct the geometric information of the point cloud according to the received occupancy map information and auxiliary information. For the specific reconstruction process, refer to the reconstruction of the point cloud reconstruction module 111 in the encoder 100 The process will not be repeated here.
  • the geometric information of the reconstructed point cloud is filtered by the point cloud filtering module 206, it is sent to the texture information reconstruction module 207 of the point cloud.
  • the point cloud texture information reconstruction module 207 is used to reconstruct the point cloud texture information to obtain a reconstructed point cloud.
  • the occupancy graph filtering module 208 is located between the occupancy graph decoding module 203 and the point cloud reconstruction module 205, and is used for filtering the occupancy graph represented by the occupancy graph information sent by the occupancy graph decoding module 203, and filtering the obtained occupancy graph.
  • the information is sent to the point cloud reconstruction module 205.
  • the occupancy map here is an occupancy map filled with point clouds.
  • the occupancy map filtering module 208 may include a classification submodule, a preprocessing submodule, and a filtering submodule. For the steps performed by each sub-module, reference may be made to the description of the corresponding sub-module in the occupancy graph filtering module 112 above.
  • the occupancy graph filtering module 208 is also connected to the auxiliary information decoding module 204, as shown by the dotted line in FIG.
  • the occupancy graph filtering module 208 may be used to receive the identification information of the target processing manner obtained by the auxiliary information decoding module 204 parsing the code stream.
  • the occupancy graph filtering module 208 can also be used to receive the Block2PatchIndex obtained by the auxiliary information decoding module 204 parsing the code stream.
  • the decoder 200 shown in FIG. 4 is only an example, and in specific implementation, the decoder 200 may include more or fewer modules than those shown in FIG. 4. This embodiment of the present application does not limit this.
  • the point cloud filtering module 110 in the encoder 100 and the point cloud filtering module 206 in the decoder 200 can remove pixels with obvious noise features such as free points and frizzy boundaries in the reconstructed point cloud . That is to say, using the point cloud filtering module can remove a part of outlier points (ie, outliers or abnormal points) in the reconstructed point cloud. However, if only the outlier points in the reconstructed point cloud can be removed by the point cloud filtering module, the effect is not good.
  • the embodiments of the present application provide a new point cloud codec method and codec.
  • a filling method may include: traversing each B0*B0 pixel block in the occupancy graph of the point cloud, and there is no overlap between different B0*B0 pixel blocks. For any pixel block of B0*B0, if the value of at least one pixel in the pixel block is 1, then the values of all pixels in the pixel block are filled with 1 (that is, all are set to 1).
  • B0*B0 is the basic filling unit to perform filling.
  • the resolution of the occupancy map of the filled point cloud is B0*B0, which is described here in a unified manner, and will not be described in detail below.
  • This filling method is only an example, and does not limit the filling method applicable to the embodiments of the present application.
  • the pixels that are filled (that is, pixels with a pixel value of 0 before filling and a pixel value of 1 after filling) will become outlier points in the reconstructed point cloud after reconstruction.
  • the quantization error will cause two points in the point cloud to appear at the same position with a certain probability. , The higher the probability of two points on the point cloud appearing at the same position, which in turn makes holes appear on the boundary of the patch in the reconstructed point cloud.
  • the embodiments of the present application provide outlier points that can effectively reduce the appearance of the reconstructed point cloud due to the filling of the point cloud occupancy graph, thereby improving the performance of encoding and decoding. Specifically, before the point cloud is reconstructed at the encoding end and/or the decoding end, the filled occupancy map of the point cloud is filtered, and the point cloud is reconstructed using the filtered occupancy map of the point cloud.
  • a solution is to process the boundary pixel block based on the type of the boundary pixel block in the occupancy graph of the point cloud before reconstructing the point cloud, such as a zero-set operation, a set-one operation, or an expansion operation. Therefore, how to determine the type of boundary pixel blocks in the point cloud occupancy graph will directly affect the difference between the processing result and the real (ie, unfilled) point cloud occupancy graph, thereby affecting the performance of codec.
  • the method for determining the type of the boundary pixel block in the occupancy graph of the point cloud in the point cloud decoding method can also be applied to other scenarios.
  • any of the following point cloud coding methods may be performed by the source device 10 in the point cloud decoding system, and more specifically, by the source device 10 is performed by the encoder 100; any of the following point cloud decoding methods may be performed by the destination device 20 in the point cloud decoding system, more specifically, by the decoder 200 in the destination device 20 .
  • the point cloud decoding method described below may include a point cloud encoding method or a point cloud decoding method if no description is given.
  • the point cloud decoding method is specifically a point cloud encoding method
  • the point cloud to be decoded in the embodiment shown in FIG. 5 is specifically a point cloud to be encoded;
  • the point cloud decoding method is specifically a point cloud decoding method, the figure
  • the point cloud to be decoded in the embodiment shown in 5 is specifically a point cloud to be decoded.
  • FIG. 5 it is a schematic flowchart of a point cloud decoding method provided by an embodiment of the present application.
  • the method may include:
  • the invalid pixel block includes pixel blocks whose value of the contained pixels is not all 0 and does not belong to the target patch occupancy map; or, the invalid pixel block includes pixel blocks whose value of the contained pixels are all 0, and the contained pixels The value of is not all 0 and does not belong to the pixel block of the target patch occupancy map; the target patch occupancy map is the patch occupancy map to which the boundary pixel block to be processed belongs.
  • the occupancy map of the point cloud to be decoded in the embodiment of the present application may refer to the occupancy map before the point cloud to be decoded is filled, or may refer to the occupancy map after the point cloud to be decoded is filled.
  • the pixel blocks in the occupied map of the point cloud to be decoded are divided into valid pixel blocks and invalid pixel blocks. Further, the invalid pixel blocks are divided into empty invalid pixel blocks and non-empty invalid pixel blocks.
  • the empty and invalid pixel block is a pixel block in which the values of the included pixels are all 0.
  • a non-empty invalid pixel block is a pixel block whose contained pixels are not all 0 and do not belong to the target patch occupancy map.
  • the target patch occupancy map is the patch occupancy map to which the boundary pixel block to be processed belongs. Correspondingly, the pixel block of the contained pixel value is not all 0 and belongs to the target patch occupancy map as the effective pixel block.
  • non-empty invalid pixel blocks in the embodiments of the present application are relative to the boundary pixel block to be processed. Therefore, there may be a boundary pixel block 1 to be processed relative to the boundary pixel block to be processed in the occupancy map of the point cloud to be decoded. A certain pixel block is an invalid pixel block; relative to the boundary pixel block 2 to be processed, this pixel block is a valid pixel block.
  • the spatial neighboring pixel blocks of a pixel block include those adjacent to the pixel block and located directly above, directly below, directly to the left, right, top, left, bottom, right, and bottom right of the pixel block One or more pixel blocks.
  • the decoder may determine whether the two pixel blocks are adjacent according to the coordinates of the two pixel blocks, and the orientation of one pixel block of the two pixel blocks relative to the other pixel block.
  • the pixel block to be processed is a non-boundary pixel block of the occupancy map of the point cloud to be decoded. Otherwise, the pixel block is the boundary pixel block of the occupancy map of the point cloud to be decoded.
  • the boundary pixel block to be processed may be any boundary pixel block in the occupied map of the point cloud to be decoded.
  • the boundary pixel block to be processed may be a boundary pixel block satisfying a certain condition in the occupancy map of the point cloud to be decoded, for example, when the current boundary of the occupancy map of the point cloud to be decoded
  • a preset threshold that is, the number of valid pixel blocks is less than or equal to the threshold
  • the current boundary pixel is determined
  • the block is a boundary pixel block to be processed.
  • the preset threshold may be 6, of course, the embodiments of the present application are not limited thereto.
  • the to-be-processed boundary pixel block is a basic filling unit that performs filling for the occupancy map of the point cloud to be decoded.
  • the embodiments of the present application are not limited thereto.
  • the embodiment of the present application does not limit how to determine the patch occupancy map to which a pixel block belongs. For example, reference may be made to the prior art.
  • the embodiments of the present application provide the following technical solutions:
  • the correspondence between the coding block where the pixel block is located and the patch is obtained, so that according to the correspondence, the patch occupancy map to which the pixel block belongs is determined.
  • the coding block is the basic unit of packaging.
  • one coding block is larger than or equal to the pixel block, for example, the coding block is usually larger than or equal to the pixel block, and the size of the coding block is usually an integer multiple of the size of the pixel block.
  • the size of the encoding block is 16x16
  • the size of the pixel block is 4x4.
  • a patch occupancy graph will occupy multiple coding blocks, and a coding block can only be occupied by one patch.
  • Block2PatchIndex needs to be used to indicate that each coding block is occupied by which patch of the multiple patches where the bounding box overlaps at the coding block , Such as recording the correspondence between the number of the encoding block and the number of the patch.
  • the patch number corresponds to the patch occupancy number. Therefore, the number of the patch occupancy map where the coding block is located can be obtained based on Block2PatchIndex, thereby obtaining the number of the patch occupancy map where the relevant pixel block is located. Since the auxiliary information is generally losslessly encoded, the Block2PatchIndex obtained by the encoder and the decoder is the same.
  • the decoder may perform S101 in units of pixel blocks. Specifically: For the i-th pixel block, first determine the patch occupancy map corresponding to the coding block where the boundary pixel block to be processed is based on Block2PatchIndex, and use the patch occupancy map as the target patch occupancy map and the boundary pixel block to be processed The patch occupancy map corresponding to the coding block where the adjacent pixel block of the non-empty airspace is located, so as to determine whether the pixel block to be processed is a boundary pixel block to be processed; then, according to the target patch occupancy map, the type of the boundary pixel block to be processed is determined. Among them, 1 ⁇ i ⁇ I, i and I are integers.
  • I is the number of pixel blocks in the occupied map of the point cloud to be decoded.
  • the i-th pixel block in this implementation mode can be replaced with the i-th pixel block containing pixels with non-zero values; accordingly, the meaning of I can be replaced with The number of pixel blocks with a value of 0.
  • the decoder may execute S101 in units of patch occupancy maps (or patches). Specifically: For the jth patch occupancy graph, determine the coding block it occupies based on Block2PatchIndex, and determine the pixel blocks in each coding block in sequence (specifically, the pixel block that contains non-zero-value pixels) Whether it is a boundary pixel block to be processed, and according to the jth patch occupancy map (that is, the target patch occupancy map), determine the type of the boundary pixel block to be processed. Among them, 1 ⁇ j ⁇ J, j and J are both integers. j is the number of patch occupancy maps in the occupancy graph of the point cloud to be decoded, that is, the number of patches included in the point cloud to be decoded.
  • S102 Reconstruct the point cloud to be decoded according to the type of the boundary pixel block to be processed.
  • the specific implementation can refer to the following.
  • the point cloud to be compressed is divided into multiple patches, and then the patch is mapped to the patch occupancy map, and the point cloud coding and decoding method for packaging each patch occupancy map will cause a large reconstruction error near the patch boundary. Therefore, in order to ensure the quality of the reconstructed point cloud, it is usually necessary to perform special processing on the boundary of the patch occupancy graph, such as filtering and hole filling.
  • the type of the boundary pixel block to be processed is determined based on whether the spatial adjacent boundary pixel block of the boundary pixel block to be processed is an invalid pixel block, so that the point cloud is reconstructed based on the type.
  • the invalid pixel block includes pixel blocks whose values of the included pixels are not all 0 and do not belong to the target patch occupancy map. In this way, it is helpful to identify the boundary pixel blocks with large reconstruction errors in the occupancy map of the point cloud to be decoded, so that different processing methods can be adopted to ensure the quality of the reconstructed point cloud, so the system can be improved Codec performance.
  • the above S101 may include the following implementation manners:
  • Method A Determine the position information of the invalid pixels (or valid pixels) in the boundary pixel block to be processed in the boundary pixel block to be processed based on whether the spatial adjacent pixel blocks of the boundary pixel block to be processed are invalid pixel blocks;
  • the type of boundary pixel block to be processed corresponds to different orientation information. For example, first, in the filled occupancy map of the point cloud, the spatial neighboring pixel blocks of the boundary pixel block to be processed are obtained, and then, by determining whether these spatial neighboring pixel blocks are invalid pixel blocks (or whether they are valid pixel blocks ), determine the type of boundary pixel block to be processed.
  • S101 may include: determining whether to process the boundary pixel block based on whether the spatial neighboring pixel block of the pixel block before filling is an invalid pixel block The orientation information of the invalid pixels in the boundary pixel block in the boundary pixel block to be processed. For example, first obtain the pixel block of the boundary pixel block to be processed before filling and the adjacent pixel block of the pixel block of the boundary pixel block before filling in the occupancy map of the point cloud before filling, and then determine Whether these adjacent spatial pixel blocks are invalid pixel blocks (or whether they are valid pixel blocks) determines the type of boundary pixel blocks to be processed.
  • the position information of the invalid pixels in the to-be-processed boundary pixel block in the to-be-processed boundary pixel block may include at least one of the following: directly above, directly below, directly to the left, right to the right, left above, left below, right above, and right Below.
  • orientation information of the invalid pixels in the boundary pixel block to be processed is directly above the boundary pixel block to be processed, the orientation information of the effective pixels in the boundary pixel block to be processed is directly below the boundary pixel block to be processed; If the orientation information of the invalid pixels in the boundary pixel block to be processed in the boundary pixel block to be processed is upper right, the orientation information of the effective pixels in the boundary pixel block to be processed in the boundary pixel block to be processed is lower left. Other examples are similar to this, and I will not list them one by one here.
  • the orientation information in this application refers to the orientation information of the invalid pixels in the boundary pixel block to be processed in the boundary pixel block to be processed, which will be described here in a unified manner and will not be described in detail below.
  • boundary pixel blocks to be processed correspond to different orientation information. For example, if the invalid pixels in the boundary pixel block to be processed are directly above the boundary pixel block to be processed, the type of the boundary pixel block to be processed may be marked as type A. As another example, if the invalid pixels in the boundary pixel block to be processed are directly above and below the boundary pixel block to be processed, the type of the boundary pixel block to be processed may be marked as type B. For another example, if the invalid pixels in the boundary pixel block to be processed are directly above, to the left, and right below the boundary pixel block to be processed, the type of the boundary pixel block to be processed may be marked as type C. Other examples are not listed one by one.
  • the adjacent pixel block in the spatial domain with the preset orientation of the boundary pixel block to be processed (or the pixel block before the boundary pixel block to be filled) is an invalid pixel block, it is determined that the invalid pixel in the boundary pixel block to be processed is obtained
  • the preset orientation in the boundary pixel block to be processed is one or a combination of at least two of right above, right below, right left, right right, top left, top right, bottom left, and bottom right.
  • the pixel block of the preset orientation of the boundary pixel block to be processed is an invalid pixel block, it means that the probability of the pixel of the preset orientation inside the boundary pixel block to be processed is an invalid pixel, and the pixel larger than the preset orientation is The probability of an effective pixel. Therefore, the pixel in the preset orientation determined by the decoder in the embodiment of the present application is an invalid pixel.
  • Method B Based on whether the spatial adjacent pixel block of the boundary pixel block to be processed is an invalid pixel block, determine the orientation information of the invalid pixel in the boundary pixel block to be processed in the boundary pixel block to be processed; based on the boundary pixel block to be processed is Whether the adjacent pixel block in the air domain of the invalid pixel block is an empty invalid pixel block and the determined orientation information determine the type of the boundary pixel block to be processed.
  • the occupancy map of the point cloud to be decoded includes the first to-be-processed boundary pixel block and the second to-be-processed boundary pixel block; the type of the first to-be-processed boundary pixel block is the first type, the second The type of the boundary pixel block to be processed is the second type; the orientation information corresponding to the first type and the second type is the same; and the predetermined orientation (such as at least one preset) of the spatial adjacent pixel blocks of the first boundary pixel block to be processed
  • the invalid pixel block of (azimuth) is an empty invalid pixel block, and the invalid pixel block of the preset orientation in the adjacent pixel block in the air domain of the second boundary image to be processed is a non-empty invalid pixel block.
  • boundary pixel blocks there may be cases where different types of boundary pixel blocks correspond to the same type of orientation information.
  • the difference between the different types of boundary pixel blocks is that the types of invalid pixel blocks with the same preset orientation in the adjacent pixel blocks in their respective spatial domains are different.
  • the technical solution provided by the method B is based on the finer granularity to classify the boundary pixel blocks in the point cloud occupancy graph. Therefore, in the process of reconstructing the point cloud, there are more options for processing the boundary pixel blocks, which helps to improve the performance of codec.
  • the above S102 may include the following steps S102A to S102B:
  • the target operation may include a set-zero operation, a set-one operation, or an expansion operation.
  • the expansion operation may be an expansion operation in computer vision.
  • the basic unit of the dilation operation is less than or equal to the basic unit where the pixel value of the point cloud occupancy map to be decoded is set to 1.
  • S102B Reconstruct the point cloud to be decoded according to the processed occupancy map, and the processed occupancy map includes pixel blocks that have been operated on by the target. For example, perform video decoding according to the encoded depth map to obtain the decoded depth map of the point cloud, and use the decoded depth map, the processed occupancy map of the point cloud and the auxiliary information of each patch to obtain the reconstructed point cloud geometry information.
  • Method 1 When the target operation is a zero-setting operation, according to the type of the boundary pixel block to be processed, the value of the pixel at the target position of the boundary pixel block to be processed is set to 0 according to the type of the target pixel block to be processed to obtain the pixel block set to 0 Different processing methods correspond to different target positions.
  • This way 1 can be applied to the scenario where the occupancy map of the point cloud to be decoded is a filled occupancy map.
  • the decoder performs filtering (or smoothing) of the filled occupancy map of the point cloud to be decoded before reconstructing the point cloud to be decoded.
  • filtering or smoothing
  • Method 2 When the target operation is a set-one operation, according to the type of the boundary pixel block to be processed, the value of the pixel at the target position of the boundary pixel block to be processed is set to 1 by using the corresponding target processing method to obtain the set pixel block . Different processing methods correspond to different target positions.
  • Method 3 When the target operation is an expansion operation, according to the type of the boundary pixel block to be processed, a corresponding target processing method is used to perform an expansion operation on the boundary pixel block to be processed to obtain a pixel block after the expansion operation.
  • the radius of the convolution kernel is different in different processing methods.
  • Method 2 and method 3 can be applied to the scenario where the occupancy map of the point cloud to be decoded is the occupancy map before filling.
  • the target position or performing the dilation operation conditionally it is helpful to set the value of the invalid pixels in the point cloud occupancy map to 1 and add a part of outlier points.
  • the added outlier points can be scaled at a certain scale. Filter out, and at the same time, it can fill the holes that appear on the patch boundary of the reconstructed point cloud, which solves the problem of holes appearing on the patch boundary when the point cloud is reconstructed.
  • the target operation is set to 0 operation.
  • the target position is in the boundary pixel block to be processed, and the position of the invalid pixel whose distance from the target effective pixel is greater than or equal to a preset threshold is located; or, the target position is in the boundary pixel block to be processed And the distance from the line where the target valid pixel is located is greater than or equal to the preset threshold where the invalid pixel is located.
  • the line where the target effective pixel is located is related to the type of the boundary pixel block to be processed. For specific examples, refer to the following.
  • the target effective pixel refers to the effective pixel with the furthest distance from the effective pixel boundary.
  • the effective pixel boundary is the boundary between the effective pixel and the invalid pixel.
  • the target effective pixel in the to-be-processed boundary pixel block is the bottom row of pixels in the to-be-processed boundary pixel block.
  • FIG. 6 it is a schematic diagram of a target position applicable to this example.
  • the pixel block to be processed is 4*4, and the preset threshold is 2 (specifically, 2 unit distances, one of which is between two adjacent pixels in the horizontal or vertical direction) The distance) is described as an example.
  • the target effective pixel in the to-be-processed boundary pixel block is the top-right one or more pixels in the to-be-processed boundary pixel block.
  • FIG. 7 it is a schematic diagram of a target position applicable to this example. Among them, (a) in FIG. 7 is a case where the target position is in the border pixel block to be processed, and the distance between the line and the target effective pixel is greater than or equal to the preset threshold. For illustration, (b) in FIG.
  • the boundary pixel block to be processed is a 4*4 pixel block
  • the preset threshold is 2 (specifically, 2 unit distances, one of which is two adjacent pixels in a 45-degree diagonal direction) the distance between).
  • the orientation information of the invalid pixels in the boundary pixel block to be processed is directly above and below the left
  • the orientation information of the effective pixels in the boundary pixel block to be processed is positive Bottom and top right
  • the target valid pixels in the boundary pixel block to be processed are the bottom row of pixels in the boundary pixel block to be processed, and one or more pixels at the top right, as shown in Figure 8 (a) The shaded part of the figure.
  • the preset pixel position is shown in the black part in (b) in FIG. 8.
  • the spatial neighboring pixel blocks described herein refer to the spatial neighboring pixel blocks on which the type of the boundary pixel block to be processed is determined. It should not be understood as the neighboring pixel blocks in the spatial domain of the boundary pixel block to be processed. For example, there may be a spatial adjacent pixel block of a boundary pixel block to be processed including 8 pixel blocks, but based on the following situation, only based on the top, bottom, left, and right of the boundary pixel block to be processed To determine the type of boundary pixel block to be processed. Other examples are similar to this, and they are not listed here one by one.
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: pixel blocks adjacent to the boundary pixel block to be processed and located directly above, directly below, to the left and to the right of the boundary pixel block to be processed.
  • the position information of the invalid pixels in the boundary pixel block to be processed in the boundary pixel block to be processed may include any one of the following:
  • Method 1A If the neighboring pixel blocks in the airspace in the preset direction of the boundary pixel block to be processed are invalid pixel blocks, and the other neighboring pixel blocks in the airspace are all valid pixel blocks, the invalid pixels in the boundary pixel block to be processed are waiting to be processed
  • the orientation information in the boundary pixel block is: the invalid pixel in the boundary pixel block to be processed is located in the preset direction in the boundary pixel block to be processed; the preset direction includes one of the top, bottom, left, and right One or a combination of at least two.
  • the type of the boundary pixel block to be processed corresponding to the orientation information described in manner 1A may be referred to as type 1. If the preset direction is directly below, the type of the boundary pixel block to be processed corresponding to the orientation information described in manner 1A may be referred to as type 2. If the preset direction is positive left, the type of the boundary pixel block to be processed corresponding to the orientation information described in manner 1A may be referred to as type 7. If the preset direction is to the right, the type of the boundary pixel block to be processed corresponding to the orientation information described in manner 1A may be referred to as type 8.
  • Method 1B If the pixel blocks directly above and to the right of the boundary pixel block to be processed are invalid pixel blocks, and the pixel blocks directly below and to the left of the boundary pixel block to be processed are valid pixel blocks, then the boundary pixel block to be processed
  • the orientation information of the invalid pixels in the boundary pixel block to be processed is: the invalid pixels in the boundary pixel block to be processed are located at the upper right in the boundary pixel block to be processed.
  • the type of boundary pixel block to be processed corresponding to the orientation information is called type 3.
  • the The orientation information of the invalid pixel in the boundary pixel block to be processed is: the invalid pixel in the boundary pixel block to be processed is located at the lower left of the boundary pixel block to be processed.
  • the type of boundary pixel block to be processed corresponding to the orientation information is called type 4.
  • the The orientation information of the invalid pixel in the boundary pixel block to be processed is: the invalid pixel in the boundary pixel block to be processed is located at the upper left of the boundary pixel block to be processed.
  • the type of boundary pixel block to be processed corresponding to the orientation information is called type 5.
  • the The orientation information of the invalid pixel in the boundary pixel block to be processed is: the invalid pixel in the boundary pixel block to be processed is located at the lower right of the boundary pixel block to be processed.
  • the type of boundary pixel block to be processed corresponding to the orientation information is called type 6.
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: adjacent to the boundary pixel block to be processed and located directly above, below, to the left, to the right, to the upper left, Pixel blocks at the top right, bottom left, and bottom right.
  • the neighboring spatial pixel blocks in the preset direction of the boundary pixel block to be processed are invalid pixel blocks and the other neighboring pixel blocks in the spatial domain are all valid pixel blocks, the invalid pixels in the boundary pixel block to be processed are waiting for
  • the orientation information in the processing boundary pixel block is: the invalid pixels in the boundary pixel block to be processed are located in a preset direction in the boundary pixel block to be processed; the preset direction includes upper left, upper right, lower left, or lower right.
  • the type of the boundary pixel block to be processed corresponding to the orientation information may be referred to as type 9. If the preset direction is the lower left, the type of the boundary pixel block to be processed corresponding to the orientation information may be referred to as type 10. If the preset direction is the upper left, the type of the boundary pixel block to be processed corresponding to the orientation information may be referred to as type 11. If the preset direction is the lower right, the type of the boundary pixel block to be processed corresponding to the orientation information may be referred to as type 12.
  • each small square in FIG. 9 represents a pixel block
  • the pixel block marked with the five-pointed star in the center represents the boundary pixel block to be processed
  • the black marked pixel block represents the invalid pixel block
  • the white marked pixel block represents the valid Pixel blocks.
  • Pixel blocks marked with diagonal hatching represent valid pixel blocks or invalid pixel blocks.
  • the discrimination method diagram in the first row of the table shown in FIG. 9 indicates that when the pixel block directly above the adjacent pixel block in the spatial domain of the boundary pixel block to be processed is an invalid pixel block, and directly below and left When both the square and the right-hand pixel blocks are valid pixel blocks, it is determined that the type of the boundary pixel block to be processed is type 1.
  • the schematic diagram in this row indicates that the spatial adjacent pixel blocks of the boundary pixel block to be processed have the following characteristics: the pixel block directly above is an invalid pixel block, and the pixel blocks directly below, left and right are valid pixel blocks ; And the pixel blocks in the upper left, upper right, lower left, and lower right are valid pixel blocks or invalid pixel blocks. Other examples are similar to this, and I will not list them one by one here.
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include pixel blocks adjacent to the boundary pixel block to be processed and located at the upper left, upper right, lower left and lower right of the boundary pixel block to be processed.
  • the neighboring spatial pixel blocks in the preset direction of the boundary pixel block to be processed are invalid pixel blocks and the other neighboring pixel blocks in the spatial domain are all valid pixel blocks, the invalid pixels in the boundary pixel block to be processed are waiting for
  • the orientation information in the processing boundary pixel block is: the invalid pixels in the boundary pixel block to be processed are located in the preset direction in the boundary pixel block to be processed; the preset direction includes one of upper left, upper right, lower left, and lower right or At least two.
  • p[i] in the following indicates the i-th boundary pixel block in the occupied map filled with the point cloud to be decoded
  • the index is j.
  • the encoder and the decoder use the same method to process the boundary block to be processed.
  • the specific implementation of the target position based on the type of the boundary pixel block to be processed may include:
  • FIG. 10 it is a schematic diagram of a pixel for determining a target position provided by an embodiment of the present application.
  • the pixel at the target position may be the pixel with the number ⁇ 1 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel numbered ⁇ 1 ⁇ , ⁇ 1,2, ⁇ or ⁇ 1,2,3 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the number of the boundary pixel block to be processed ⁇ 1 ⁇ , ⁇ 1,2, ⁇ , ⁇ 1,2,3 ⁇ , ⁇ 1,2,3,4 ⁇ , ⁇ 1, 2, 3, 4, 5 ⁇ , ⁇ 1, 2, 3, 4, 5, 6 ⁇ or ⁇ 1,2, 3, 4, 5, 6, 7 ⁇ .
  • the pixel at the target position may be the pixel with the number ⁇ 2 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel numbered ⁇ 4 ⁇ , ⁇ 3, 4 ⁇ or ⁇ 2, 3, 4 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the number in the boundary pixel block to be processed ⁇ 7 ⁇ , ⁇ 6, 7 ⁇ , ⁇ 5, 6, 7 ⁇ , ⁇ 4, 5, 6, 7 ⁇ , ⁇ 3, 4, 5, 6, 7 ⁇ , ⁇ 2, 3, 4, 5, 6, 7 ⁇ or ⁇ 1,2, 3, 4, 5, 6, 7 ⁇ .
  • FIG. 11 it is a schematic diagram of a pixel for determining a target position provided by an embodiment of the present application.
  • the pixel at the target position may be a pixel numbered ⁇ 1 ⁇ , ⁇ 1,2, ⁇ or ⁇ 1,2,3 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel numbered ⁇ 3 ⁇ , ⁇ 2, 3 ⁇ or ⁇ 1,2, 3 ⁇ in the boundary pixel block to be processed.
  • FIG. 12 it is a schematic diagram of a pixel for determining a target position provided by an embodiment of the present application.
  • the pixel at the target position may be a pixel numbered ⁇ 1 ⁇ , ⁇ 1,2, ⁇ or ⁇ 1,2,3 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel numbered ⁇ 3 ⁇ , ⁇ 2, 3 ⁇ or ⁇ 1,2, 3 ⁇ in the boundary pixel block to be processed.
  • FIG. 13 it is a schematic diagram of a pixel for determining a target position provided by an embodiment of the present application.
  • the pixel at the target position may be the pixel with the number ⁇ 2 ⁇ or ⁇ 1, 2 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be a pixel numbered ⁇ 4 ⁇ , ⁇ 3, 4 ⁇ ... or ⁇ 1, 2...4 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel numbered ⁇ 8 ⁇ , ⁇ 7, 8 ⁇ ... or ⁇ 1, 2...8 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel with the number ⁇ 1 ⁇ or ⁇ 1, 2 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be the pixel with the number ⁇ 1 ⁇ , ⁇ 1,2, or ⁇ 1,2,...4 ⁇ in the boundary pixel block to be processed.
  • the pixel at the target position may be a pixel numbered ⁇ 1 ⁇ , ⁇ 1,2,..., or ⁇ 1, 2...8 ⁇ in the boundary pixel block to be processed.
  • the target operation is set to 1 operation.
  • the target position is in the to-be-processed boundary pixel block, and the position of the invalid pixel whose distance from the target effective pixel is less than or equal to a preset threshold is located; or, the target position is in the to-be-processed boundary pixel block And the distance from the line where the target valid pixel is located is less than or equal to the preset threshold.
  • the line where the target effective pixel is located is related to the type of the boundary pixel block to be processed. For specific examples, refer to the following.
  • the target effective pixel refers to the effective pixel with the furthest distance from the effective pixel boundary.
  • the effective pixel boundary is the boundary between the effective pixel and the invalid pixel.
  • the target effective pixel in the to-be-processed boundary pixel block is the bottom row of pixels in the to-be-processed boundary pixel block.
  • FIG. 14 it is a schematic diagram of a target position applicable to this example.
  • the boundary pixel block to be processed is a 4*4 pixel block
  • the preset threshold is 2 (specifically, 2 unit distances, one of which is between two adjacent pixels in the horizontal or vertical direction) The distance) is described as an example.
  • the target effective pixels in the boundary pixel block to be processed are the first row pixels and the second row pixels in the a diagram in FIG. 14, and the target position in the boundary pixel block to be processed is processed using the corresponding target processing method.
  • Set the value of the pixel of 1 to get the set boundary pixel block Set the value of the pixel in the second row of Figure a in Figure 14 to 1, and set the boundary pixel block of Figure 1 as shown in Figure 14 b;
  • the values of the pixels in the first row and the second row of the a diagram in FIG. 14 are both set to 1, and the boundary pixel block set to 1 is shown in the c diagram in FIG. 14.
  • the target effective pixel in the to-be-processed boundary pixel block is the top-right one or more pixels in the to-be-processed boundary pixel block.
  • FIG. 15 it is a schematic diagram of a target position applicable to this example. Among them, (a) in FIG. 15 is a case where the target position is in the to-be-processed boundary pixel block and the distance between the straight line where the target valid pixel is located is less than or equal to the preset threshold.
  • the (e) diagram in FIG. 15 is based on the example where the target position is in the to-be-processed boundary pixel block and the distance from the target effective pixel to the invalid pixel less than or equal to the preset threshold is taken as an example. of.
  • the boundary pixel block to be processed is a pixel block with a size of 4*4
  • the preset threshold is 2 (specifically, 2 unit distances, one of which is two adjacent two in the direction of a 45-degree diagonal line) Distance between pixels).
  • FIG. 15 is an example where the target position is in the boundary pixel block to be processed, and the position of the invalid pixel whose distance from the line where the target effective pixel is located is less than or equal to 2 is taken as an example.
  • the set boundary pixel block is shown in Figure 15 (B), (c) and (d) in Figure;
  • Figure (e) in Figure 15 is based on the target position in the boundary pixel block to be processed, and the distance from the target effective pixel is less than or The position of the invalid pixel equal to 2 will be described as an example.
  • the set boundary pixel block is as shown in Figure 15 ( f) and (g).
  • the target invalid pixel in the boundary pixel block to be processed is the bottom reciprocal in the boundary pixel block to be processed
  • the pixels in the second row and one or more pixels in the upper right are shown in the shaded part of (a) in FIG. 16.
  • the target position is shown in the white part in (b) in FIG. 16.
  • the target operation is a set-one operation
  • the specific implementation manner of the type of boundary pixel block to be processed can be referred to the corresponding specific implementation manner when the target operation is a set-to-zero operation, which will not be repeated here.
  • the target position when the target operation is a zero-setting operation, includes: in the boundary pixel block to be processed, the position from the farthest distance Lmax from the target effective pixel to the target effective pixel Set the position of distance L1. Lmax>L1.
  • the target position when the target operation is set to 1, includes: the boundary pixel block to be processed, from the closest distance to the target effective pixel Lmin (such as the distance can be 0 or 1, etc.) to the target effective pixel Preset the position of distance L2. Lmin ⁇ L1.
  • the target operation is an expansion operation.
  • the dilation operation will be described by taking an example that the basic unit is one pixel.
  • the convolution kernel of the expansion operation can have any shape and size, such as a square or a circle.
  • the convolution kernel generally defines an anchor point, which is generally the center point of the convolution kernel.
  • the convolution kernel may be any one in FIG. 17.
  • white squares represent pixels with a pixel value of 0, shaded squares represent pixels with a pixel value of 1, and the pixel block where the five-pointed star is located is an anchor point.
  • the pixel p[x][y] in the occupancy diagram can be taken, and one of the convolution kernels in FIG. 17 (which one can be predefined by the encoder and decoder, of course, this application is implemented (The example is not limited to this).
  • the anchor point is aligned with p[x][y]. If the position indicated by the shaded square in the convolution kernel is at least one pixel in the neighborhood point corresponding to the p[x][y] pixel If the value is 1, q[x][y] takes the value 1, otherwise q[x][y] takes the value 0.
  • the radius of the convolution kernel determines how many pixels the dilation operation affects. The larger the radius of the convolution kernel, the more pixels are expanded; the smaller the radius of the convolution kernel, the fewer pixels are expanded.
  • the target operation is usually set to 0 as an example.
  • the above S102A may include: determining a processing method corresponding to the type of the boundary pixel block to be processed according to the mapping relationship between various types of boundary pixel blocks and multiple processing methods; if the type of the boundary pixel block to be processed corresponds One processing method, the processing method corresponding to the type of the boundary pixel block to be processed is taken as the target processing method; or, if the type of the boundary pixel block to be processed corresponds to multiple processing methods, the type corresponding to the type of the boundary pixel block to be processed is One of the multiple processing methods is used as the target processing method; the target processing method is used to perform the target operation on the boundary pixel block to be processed to obtain the pixel block subjected to the target operation.
  • the encoder and decoder can predefine (eg, by protocol) the mapping relationship between various types of boundary pixel blocks and multiple processing methods, for example, the number of predefined boundary pixel blocks The mapping relationship between various types of identification information and identification information of multiple processing methods.
  • the specific embodiment of the above mapping relationship is not limited.
  • it may be a table, a formula, or a logical judgment based on conditions (such as if else or switch operations, etc.).
  • the specific embodiment of the mapping relationship is mainly used as an example for description. Based on this, when executing S102A, the decoder can obtain the processing method corresponding to the type of the boundary pixel block to be processed by looking up the table.
  • the above mapping relationship is specifically embodied in one or more tables, which is not limited in this embodiment of the present application. For ease of description, the embodiments of the present application are described by taking these tables embodied in a table as an example.
  • the above S102A may specifically include: looking up a table according to the type of the boundary pixel block to be processed, to obtain a processing method corresponding to the type of the boundary pixel block to be processed, the table includes between the types of the boundary pixel block and the various processing methods Mapping relationship.
  • both the encoder and the decoder can obtain the target processing method through the predefined mapping relationship described above. Therefore, in this case, the encoder does not need to send the identification information indicating the target processing mode to the decoder, which can save code stream transmission overhead.
  • a processing method corresponding to the type may be: The pixel with the number ⁇ 1 ⁇ in the processing boundary pixel block is set to 0.
  • the encoder may select one processing method from the multiple processing methods as the target processing method. For example, according to the position of the pixel whose pixel value is 0 in the pixel block before being filled in the boundary pixel block to be processed, one processing method is selected as the target processing method from various processing methods corresponding to the type of the boundary pixel block to be processed. For example, according to the description above, based on FIG.
  • the multiple processing methods corresponding to this type may be: the number in the boundary pixel block to be processed is ⁇ 1 ⁇ Is set to 0, and the pixel with the number ⁇ 1, 2 ⁇ in the boundary pixel block to be processed is set to 0.
  • the target processing method may be to set the pixel number ⁇ 1 ⁇ in the boundary pixel block to be processed to 0, or to set the pixel number ⁇ 1, 2 ⁇ in the boundary pixel block to be processed to 0.
  • one of the processing methods corresponding to the multiple processing methods of the type of the boundary pixel block to be processed as the target processing method may include: according to the pixel value of the boundary pixel block to be processed in the pixel block before filling, the pixel value is 0 For the position of the pixel, select a processing method from the multiple processing methods corresponding to the type of boundary pixel block to be processed as the target processing method. Among them, the selected target processing mode makes the most invalid pixels in the boundary pixel block to be processed set to 0.
  • FIG. 18 it is a schematic diagram of pixel blocks before filling two types of boundary pixel blocks to be processed (that is, invalid pixels directly above the boundary pixel blocks to be processed) provided by the embodiment of the present application.
  • the target processing method may be to number the boundary pixel block to be processed as The pixel of ⁇ 1 ⁇ is set to 0. If the boundary pixel block to be processed is shown in (b) in FIG.
  • the target processing method may be to convert the boundary pixel block to be processed
  • the pixel numbered ⁇ 1, 2 ⁇ is set to 0.
  • the size of the boundary pixel block to be processed is 4*4 as an example. The principles of other examples are similar to this, and will not be repeated here.
  • the encoder may encode the identification information into the code stream, where the identification information indicates the target processing method of the boundary pixel block to be processed.
  • the above S102A may include: parsing the code stream according to the type of the boundary pixel block to be processed to obtain the identification information; and then performing target operations on the boundary pixel block to be processed in a target processing manner.
  • boundary pixel blocks can also be classified into other types.
  • the type with a relatively high probability of occurrence may be selected, or the encoding efficiency after performing the zero-setting process provided in the embodiments of the present application may be The type with larger gain contribution is used to execute the technical solution provided by the embodiment of the present application, and for other types, the technical solution provided by the embodiment of the present application may not be executed.
  • the type of boundary pixel block to be processed (specifically, the type of boundary pixel block that is coded according to the technical solution provided by the embodiments of the present application, or the boundary pixel corresponding to multiple processing methods) Block type) to determine whether to parse the code stream.
  • the code stream here refers to a code stream carrying identification information of the target processing mode.
  • the encoder and the decoder are pre-defined: for various types of boundary pixel blocks as shown in FIG. 9, codec is decoded according to the technical solution provided by the embodiments of the present application; then, for the decoder, when determining a When the type of the boundary pixel block to be processed is one of the types shown in FIG. 9, the code stream is parsed to obtain the target processing method corresponding to the type; when the type of the boundary pixel block to be processed is not shown in FIG. 9 When type, do not parse the code stream. In this way, there is no need to transmit each type of each to-be-processed boundary pixel block and the corresponding target processing manner of each type in the code stream, so the code stream transmission overhead can be saved.
  • the decoder may determine whether the adjacent pixel block in the target airspace of the boundary pixel block to be processed is an empty invalid pixel block, from Among the various processing methods corresponding to the type of the boundary pixel block to be processed, one processing method is selected as the target processing method; the target adjacent pixel block in the spatial domain is the adjacent pixel block in the spatial domain which is an invalid pixel block to be processed.
  • the embodiment of the present application supports determining the target position based on a preset threshold and a distance value determined by a non-empty invalid pixel block among adjacent pixel blocks in the spatial domain of the boundary pixel block to be processed.
  • a processing method is selected as the target processing method from a variety of processing methods corresponding to the type of the boundary pixel block to be processed, Including: if the adjacent pixel block of the target airspace is an empty invalid pixel block, then select the first processing method as the target processing method from a variety of processing methods corresponding to the type of boundary pixel block to be processed; if the adjacent pixel block of the target airspace is non-empty For the invalid pixel block, the second processing method is selected as the target processing method from multiple processing methods corresponding to the type of the boundary pixel block to be processed.
  • the target operation is a zero-setting operation
  • the range of the target position corresponding to the first processing mode is larger than the range of the target position corresponding to the second processing mode.
  • the target operation is a set-one operation
  • the range of the target position corresponding to the first processing mode is smaller than the range of the target position corresponding to the second processing mode.
  • the target operation is an expansion operation
  • the radius of the convolution kernel corresponding to the first processing mode is smaller than the radius of the convolution kernel corresponding to the second processing mode.
  • the adjacent pixel block in the airspace of the boundary pixel block to be processed includes a non-empty invalid pixel block
  • two different patch occupancy maps that is, the patch occupancy map where the boundary pixel block to be processed is located and the non-empty invalid pixel
  • the patch occupancy map where the block is located is relatively more encrypted and packed in the vicinity of the boundary pixel block to be processed. Therefore, the local discontinuity of the corresponding depth image and texture image here is usually larger than the boundary pixel block where there is no non-empty invalid adjacent pixel block nearby. Therefore, the coding error here will also be relatively large.
  • the zero-setting operation here is relatively more conservative (ie, a relatively small number of pixels are set to 0 in the corresponding direction), or the set-one operation is relatively more aggressive ( That is, the range of the target position corresponding to the set 1 operation is relatively larger, or the radius of the convolution kernel corresponding to the expansion operation is relatively larger).
  • the pixels in the top row of the pixel block to be processed are set to zero; If the adjacent pixel block directly above the pixel block to be processed is an empty and invalid pixel block, the pixels in the top two rows of the pixel block to be processed are set to zero.
  • the invalid pixels with a distance of 2 from the effective pixel are all set to 1; if the pixel block to be processed is directly above If the adjacent pixel block is an empty invalid pixel block, all invalid pixels with a distance of 1 from the effective pixel are set to 1.
  • the range of the target position corresponding to the third processing manner is smaller than the range of the target position corresponding to the fourth processing manner.
  • the range of the target position corresponding to the third processing mode is larger than the range of the target position corresponding to the fourth processing mode.
  • the radius of the convolution kernel corresponding to the third processing mode is larger than the radius of the convolution kernel corresponding to the fourth processing mode.
  • the third processing method is a target processing method corresponding to the type of the first pixel block to be processed
  • the fourth processing method is a target processing method corresponding to the type of the second pixel block to be processed.
  • the air-domain neighboring pixel block in the preset orientation of the first to-be-processed pixel block is an empty invalid pixel block
  • the air-domain neighboring pixel block in the preset orientation of the second to-be-processed pixel block is a non-empty invalid pixel block
  • FIG. 19 it is a schematic diagram of a code stream structure provided by an embodiment of the present application.
  • Each arrowed line in FIG. 19 indicates the correspondence between a boundary pixel block and the identification information of the target processing method of the boundary pixel block.
  • the numbers in FIG. 19 indicate the index of the boundary pixel block.
  • the encoder can dynamically determine the target processing mode corresponding to the type of boundary pixel block to be processed, and then encode the relevant information of the target processing mode into the code stream.
  • the decoder can obtain this by parsing the code stream.
  • the relevant information of the target processing mode may include: the index (such as coordinate value) of the pixel set to 0 or 1.
  • FIG. 20 it is a schematic flowchart of a point cloud coding method provided by an embodiment of the present application.
  • the execution subject of this embodiment may be an encoder.
  • the method may include:
  • S301 Determine the indication information used to indicate whether to process the occupancy map of the point cloud to be encoded according to the target encoding method;
  • the target encoding method includes any point cloud encoding method provided in the embodiments of the present application, for example, FIG. 5
  • the illustrated point cloud decoding method, and decoding here specifically refers to coding.
  • there may be at least two encoding methods one of the at least two may be any point cloud encoding method provided in the embodiments of the present application, and the other may be existing technologies or points provided in the future Cloud coding method.
  • the indication information may specifically be an index of the target point cloud encoding/decoding method.
  • the encoder and the decoder may pre-appoint the indexes of at least two point cloud encoding/decoding methods supported by the encoder/decoder, and then, after the encoder determines the target encoding method, the index of the target encoding method.
  • the index or index of the decoding method corresponding to the target encoding method is encoded into the code stream as indication information.
  • the embodiments of the present application do not limit how the encoder determines whether the target encoding method is at least one of at least two encoding methods supported by the encoder.
  • S302 Encode the instruction information into the code stream.
  • the indication information is frame level information.
  • This embodiment provides a technical solution for selecting a target encoding method.
  • the technical solution can be applied to a scenario where an encoder supports at least two point cloud encoding methods.
  • FIG. 21 it is a schematic flowchart of a point cloud decoding method provided by an embodiment of the present application.
  • the execution subject of this embodiment may be a decoder.
  • the method may include:
  • the target decoding method includes any point cloud decoding method provided in the embodiments of the present application, For example, it may be the point cloud decoding method shown in FIG. 5, and the decoding here specifically refers to decoding. Specifically, it is a decoding method corresponding to the encoding method described in FIG. 20.
  • the indication information is frame level information.
  • the point cloud decoding method provided in this embodiment corresponds to the point cloud encoding method provided in FIG. 20.
  • the above mainly introduces the solutions provided by the embodiments of the present application from the perspective of a method.
  • it includes hardware structures and/or software modules corresponding to performing each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed by hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
  • the embodiments of the present application may divide the function modules of the encoder/decoder according to the above method examples.
  • each function module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules may be implemented in the form of hardware or software function modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a division of logical functions. In actual implementation, there may be another division manner.
  • FIG. 22 it is a schematic block diagram of a decoder 230 provided by an embodiment of the present application.
  • the decoder 230 may specifically be an encoder or a decoder.
  • the decoder 230 may include a classification module 2301 and a reconstruction module 2302.
  • the classification module 2301 may correspond to the classification submodule in the occupancy graph filtering module 112 in FIG. 2, and accordingly, the reconstruction module 2302 may correspond to one in FIG. 2 Or a combination of multiple modules, for example, the reconstruction module 2302 may correspond to the preprocessing submodule in the occupancy graph filtering module 112 in FIG. 2 and the point cloud reconstruction module 111.
  • the classification module 2301 may correspond to the classification submodule in the occupancy graph filtering module 208 in FIG. 2, and accordingly, the reconstruction module 2302 may correspond to one in FIG. 4 Or a combination of multiple modules, for example, the reconstruction module 2302 may correspond to the preprocessing submodule in the occupancy graph filtering module 208 in FIG. 4 and the point cloud reconstruction module 111.
  • the classification module 2301 is used to determine the type of the boundary pixel block to be processed based on whether the spatial adjacent pixel block of the boundary pixel block to be processed of the occupancy map of the point cloud to be decoded is an invalid pixel block; wherein, the invalid pixel The block includes pixel blocks whose value of the contained pixels is not all 0 and does not belong to the target point cloud block patch occupancy map; or, the invalid pixel block includes pixel blocks whose contained values of the pixels are all 0, and the value of the contained pixels The pixel blocks whose values are not all 0 and do not belong to the target patch occupancy map; the target patch occupancy map is the patch occupancy map to which the boundary pixel block to be processed belongs.
  • the reconstruction module 2302 is configured to reconstruct the point cloud to be decoded according to the type of the boundary pixel block to be processed.
  • the classification module 2301 is also used to determine the current boundary pixel when the number of invalid pixel blocks in the spatial adjacent pixel blocks of the current boundary pixel block of the occupancy map of the point cloud to be decoded is greater than or equal to a preset threshold
  • the block is a boundary pixel block to be processed.
  • the classification module 2301 determines whether the adjacent pixel block in the air domain of the boundary pixel block to be processed based on the occupancy map of the point cloud to be decoded is an invalid pixel block, and is specifically used to: Based on whether the spatial adjacent pixel block of the boundary pixel block to be processed is an invalid pixel block, determine the position information of the invalid pixel in the boundary pixel block to be processed in the boundary pixel block to be processed; wherein, different types of boundary pixel blocks correspond to different Location information.
  • the spatial adjacent pixel block of the preset orientation of the boundary pixel block to be processed is an invalid pixel block
  • determine the preset orientation of the invalid pixel in the boundary pixel block to be processed in the boundary pixel block to be processed is one or a combination of at least two of directly above, below, above, to the left, above to the right, above left, above right, below left, and below right.
  • the reconstruction module 2302 is specifically configured to: according to the type of the boundary pixel block to be processed, adopt a corresponding target processing method to perform target operation on the boundary pixel block to be processed to obtain a pixel block subjected to the target operation, for example, this step may be considered as Performed by the preprocessing sub-module in the occupancy graph filtering module 112 in FIG. 2 or the occupancy graph filtering module 208 in FIG. 4; reconstructing the point cloud to be decoded according to the processed occupancy graph, and the processed occupancy
  • the graph includes pixel blocks that have been subjected to target operations. For example, this step may be considered to be performed by the point cloud reconstruction module 111 in FIG. 2 or the point cloud reconstruction module 205 in FIG. 4.
  • the reconstruction module 2302 performs the target operation on the boundary pixel block to be processed according to the type of the boundary pixel block to be processed according to the type of the boundary pixel block to be processed, and obtains the pixel block subjected to the target operation.
  • the corresponding target processing method when the target operation is When set to 0, according to the type of boundary pixel block to be processed, the corresponding target processing method is used to set the value of the pixel at the target position of the boundary pixel block to be processed to 0, to obtain the pixel block set to 0, the target corresponding to different processing methods The position is different; or, when the target operation is a set operation, according to the type of the boundary pixel block to be processed, the corresponding target processing method is used to set the value of the pixel at the target position of the boundary pixel block to be processed to obtain the set 1 Pixel blocks, different processing methods correspond to different target positions; or, when the target operation is an expansion operation, according to the type of boundary pixel block to be processed, the corresponding target processing method is used to perform an expansion operation on the boundary pixel block to be processed, to obtain an expanded operation After the pixel block, the radius of the convolution kernel in different processing methods is different.
  • the reconstruction module 2302 performs the target operation on the boundary pixel block to be processed according to the type of the boundary pixel block to be processed, and uses the corresponding target processing method to obtain the pixel block subjected to the target operation.
  • the processing method corresponding to the type of the block is regarded as the target processing method; or, if the type of the boundary pixel block to be processed corresponds to multiple processing methods, one of the processing methods corresponding to the type of the boundary pixel block to be processed is used as Target processing method; adopting the target processing method to perform target operation on the boundary pixel block to be processed to obtain the pixel block subjected to the target operation.
  • the reconstruction module 2302 performs target operation on the boundary pixel block to be processed according to the type of the boundary pixel block to be processed, to obtain the pixel block subjected to the target operation, and is specifically used for: according to the to-be-processed Look up the table of the types of boundary pixel blocks to obtain the processing methods corresponding to the types of boundary pixel blocks to be processed.
  • the table includes the mapping relationship between the types of boundary pixel blocks and various processing methods; if the types of boundary pixel blocks to be processed correspond One processing method, the processing method corresponding to the type of the boundary pixel block to be processed is taken as the target processing method; or, if the type of the boundary pixel block to be processed corresponds to multiple processing methods, the type corresponding to the type of the boundary pixel block to be processed is One of the multiple processing methods is used as the target processing method; the target processing method is used to perform the target operation on the boundary pixel block to be processed to obtain the pixel block subjected to the target operation.
  • the reconstruction module 2302 regards one of multiple processing modes corresponding to the type of boundary pixel block to be processed as the target processing mode, which is specifically used to: according to the target spatial phase of the boundary pixel block to be processed Whether the adjacent pixel block is an empty and invalid pixel block, select a processing method as the target processing method from a variety of processing methods corresponding to the type of the boundary pixel block to be processed; the adjacent pixel block of the target airspace is invalid for the boundary pixel block to be processed The spatial domain of the pixel block is adjacent to the pixel block.
  • the reconstruction module 2302 selects a processing method from multiple processing methods corresponding to the type of the boundary pixel block to be processed according to whether the adjacent pixel block of the target airspace of the boundary pixel block to be processed is an empty invalid pixel block
  • the aspect of the target processing method is specifically used for: if the adjacent pixel block in the target airspace is an empty invalid pixel block, then the first processing method is selected as the target processing method from a variety of processing methods corresponding to the type of boundary pixel block to be processed; if If the neighboring pixel block in the target airspace is a non-empty invalid pixel block, then the second processing method is selected as the target processing method from a variety of processing methods corresponding to the type of boundary pixel block to be processed; where, when the target operation is a zero-setting operation, the second The range of the target position corresponding to the first processing method is larger than the range of the target position corresponding to the second processing method; or, when the target operation is a set operation, the range of the target position corresponding to the first processing
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: pixel blocks adjacent to the boundary pixel block to be processed and located directly above, directly below, to the left, and to the right of the boundary pixel block to be processed; if The adjacent pixel block in the airspace in the preset direction of the boundary pixel block to be processed is an invalid pixel block, and the adjacent pixel blocks in other airspace are all valid pixel blocks, then the orientation information is: the invalid pixels in the boundary pixel block to be processed are located in the pending The preset direction in the boundary pixel block; the preset direction includes one or a combination of at least two of the top right, bottom right, front left, and front right; or, if the top and right of the boundary pixel block to be processed The square pixel block is an invalid pixel block, and the pixel blocks directly below and to the left of the boundary pixel block to be processed are valid pixel blocks, then the orientation information is: the invalid pixels in the boundary pixel block to be processed are located in the boundary pixel block to be
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include pixel blocks adjacent to the boundary pixel block to be processed and located at the upper left, upper right, lower left, and lower right of the boundary pixel block to be processed;
  • the adjacent pixel blocks in the airspace in the preset direction of the processing of the boundary pixel block are invalid pixel blocks, and the adjacent pixel blocks in the other airspace are all valid pixel blocks.
  • the orientation information is: the invalid pixels in the boundary pixel block to be processed are located at the boundary to be processed
  • the preset direction in the pixel block; the preset direction includes one or at least two of upper left, upper right, lower left, and lower right.
  • the spatial adjacent pixel blocks of the boundary pixel block to be processed include: adjacent to the boundary pixel block to be processed and located directly above, directly below, to the left, to the right, or to the left of the boundary pixel block to be processed , Upper right, lower left and lower right pixel blocks; if the adjacent pixel blocks in the airspace in the preset direction of the boundary pixel block to be processed are invalid pixel blocks, and the other adjacent pixel blocks in the airspace are all valid pixel blocks, the orientation information Yes: the invalid pixels in the boundary pixel block to be processed are located in the preset direction in the boundary pixel block to be processed; the preset direction includes upper left, upper right, lower left, or lower right.
  • the target position is in the boundary pixel block to be processed, and the invalid pixel whose distance from the target effective pixel is greater than or equal to the preset threshold is located; or, the target position is In the boundary pixel block to be processed, and the distance between the straight line where the target valid pixel is located is greater than or equal to the preset threshold; the line is related to the type of the boundary pixel block to be processed; or, when the target operation is When set to 1, the target position is in the boundary pixel block to be processed, and the invalid pixel whose distance from the target effective pixel is less than or equal to the preset threshold is located; or, the target position is in the boundary pixel block to be processed, And the distance from the straight line where the target valid pixel is located is less than or equal to the position of the invalid pixel where the preset threshold is located; the straight line is related to the type of the boundary pixel block to be processed.
  • the classification module 2301 determines whether the adjacent pixel block in the air domain of the boundary pixel block to be processed based on the occupancy map of the point cloud to be decoded is an invalid pixel block, and is specifically used to: Based on whether the spatial adjacent pixel block of the boundary pixel block to be processed is an invalid pixel block, determine the orientation information of the invalid pixel in the boundary pixel block to be processed in the boundary pixel block to be processed; the invalid pixel block based on the boundary pixel block to be processed Whether the adjacent pixel block in the air domain is an empty invalid pixel block, and the determined orientation information determines the type of the boundary pixel block to be processed.
  • the occupancy map of the point cloud to be decoded includes a first to-be-processed boundary pixel block and a second to-be-processed boundary pixel block;
  • the type of the first to-be-processed boundary pixel block is the first type, and the second to-be-processed boundary pixel block
  • the type of is the second type;
  • the first type and the second type correspond to the same orientation information;
  • the invalid pixel block with the preset orientation in the adjacent pixel block in the spatial domain of the first boundary pixel block to be processed is an empty invalid pixel block, the first 2.
  • Invalid pixel blocks with preset orientations in adjacent pixel blocks in the spatial domain of the boundary image to be processed are non-empty invalid pixel blocks.
  • the decoder 230 is an encoder. If the type of the boundary pixel block to be processed corresponds to multiple processing methods; as shown in FIG. 23, the encoder further includes: an auxiliary information encoding module 2303 for encoding the identification information Into the code stream, the identification information indicates the target processing method of the boundary pixel block to be processed.
  • the decoder 230 is a decoder. If the type of the boundary pixel block to be processed corresponds to multiple processing methods, the reconstruction module 2302 performs a target operation on the boundary pixel block to be processed by using a target processing method to obtain pixels subjected to the target operation
  • the aspect of the block is specifically used for: parsing the code stream according to the type of the boundary pixel block to be processed to obtain identification information; the identification information indicates the target processing method; adopting the target processing method indicated by the identification information to perform the target operation on the boundary pixel block to be processed To get the pixel block operated by the target.
  • the decoder 230 is a decoder.
  • the decoder further includes: auxiliary information decoding module 2304, which is used to parse the code stream to obtain the size of the boundary pixel block to be processed of the point cloud to be decoded Information; the classification module 2301 is specifically used to divide the occupancy map of the point cloud to be decoded according to the size information to obtain one or more boundary pixel blocks to be processed.
  • the encoder 250 may include an auxiliary information encoding module 2501.
  • the encoder 250 may be the encoder 100 in FIG. 2.
  • the auxiliary information encoding module 2501 may be the auxiliary information encoding module 108.
  • the auxiliary information encoding module 2501 is used to determine the indication information and encode the indication information into the code stream.
  • the indication information is used to indicate whether to process the occupancy map of the point cloud to be encoded according to the target encoding method; the target encoding method includes any of the point cloud decoding methods (specifically, the point cloud encoding method) provided above, as shown in FIG. 5 The point cloud decoding method shown.
  • the encoder 250 further includes an occupancy graph filtering module 2502 and a point cloud reconstruction module 2503, which are used to process the occupancy graph of the encoded point cloud according to the target encoding method.
  • the steps performed by the occupancy graph filtering module 2502 can refer to the steps performed by the classification module 2301 and the preprocessing submodule
  • the steps performed by the point cloud reconstruction module 2503 can refer to the steps performed by the point cloud reconstruction module, I won't repeat them here.
  • the decoder 260 may include an auxiliary information decoding module 2601, an occupancy map filtering module 2602, and a point cloud reconstruction module 2603.
  • the auxiliary information decoding module 2601 is used to parse the code stream to obtain indication information, which is used to indicate whether to process the occupancy map of the point cloud to be decoded according to the target decoding method; the target decoding method includes any of the above
  • the point cloud decoding method (specifically, the point cloud decoding method) is as shown in FIG. 5.
  • the occupancy graph filtering module 2602 and the point cloud reconstruction module 2603 are used to process the occupancy graph of the decoded point cloud according to the target decoding method when the indication information is used to instruct the occupancy graph of the decoded point cloud to be processed according to the target decoding method
  • the steps performed by the occupancy graph filtering module 2602 can refer to the steps performed by the classification module 2301 and the preprocessing submodule
  • the steps performed by the point cloud reconstruction module 2603 can refer to the steps performed by the point cloud reconstruction module, here No longer.
  • each module in the decoder 230, the encoder 250, or the decoder 260 provided in the embodiment of the present application is a functional body that implements various execution steps included in the corresponding method provided above, that is, it has The functional bodies that fully implement the various steps in the image filtering method of the present application and the expansion and deformation of these steps, please refer to the introduction of the corresponding methods above for details. For the sake of brevity, this article will not repeat them.
  • FIG. 27 is a schematic block diagram of an implementation manner of an encoding device or a decoding device (referred to simply as a decoding device 270) used in an embodiment of the present application.
  • the decoding device 270 may include a processor 2710, a memory 2730, and a bus system 2750.
  • the processor 2710 and the memory 2730 are connected through a bus system 2750.
  • the memory 2730 is used to store instructions.
  • the processor 2710 is used to execute the instructions stored in the memory 2730 to perform various point cloud decoding methods described in this application. In order to avoid repetition, they are not described in detail here.
  • the processor 2710 may be a central processing unit (central processing unit, CPU), and the processor 2710 may also be other general-purpose processors, DSP, ASIC, FPGA, or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 2730 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as the memory 2730.
  • the memory 2730 may include code and data 2731 accessed by the processor 2710 using the bus 2750.
  • the memory 2730 may further include an operating system 2733 and an application program 2735, the application program 2735 including allowing the processor 2710 to perform the video encoding or decoding method described in this application (especially the current pixel block based on the block size of the current pixel block described in this application for the current pixel block At least one program of the filtering method).
  • the application program 2735 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video decoding application) that performs the video encoding or decoding method described in this application.
  • the bus system 2750 may also include a power bus, a control bus, and a status signal bus. However, for clarity, various buses are marked as the bus system 2750 in the figure.
  • the decoding device 270 may also include one or more output devices, such as a display 2770.
  • the display 2770 may be a tactile display that combines the display with a tactile unit that operably senses touch input.
  • the display 2770 may be connected to the processor 2710 via the bus 2750.
  • Computer readable media may include computer readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (eg, according to a communication protocol).
  • computer-readable media may generally correspond to non-transitory tangible computer-readable storage media, or communication media, such as signals or carrier waves.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • Such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, flash memory, or may be used to store instructions or data structures
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave are used to transmit instructions from a website, server, or other remote source
  • coaxial cable Wire, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media.
  • magnetic disks and optical disks include compact disks (CDs), laser optical disks, optical optical disks, DVDs, and Blu-ray disks, where magnetic disks typically reproduce data magnetically, while optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • the term "processor” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.
  • the functions described in the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or in combination Into the combined codec.
  • the techniques can be fully implemented in one or more circuits or logic elements.
  • various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 may be understood as corresponding circuit devices or logic elements.
  • the technology of the present application can be implemented in a variety of devices or equipment, including wireless handsets, integrated circuits (ICs), or a set of ICs (eg, chipsets).
  • ICs integrated circuits
  • a set of ICs eg, chipsets
  • Various components, modules or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but do not necessarily need to be implemented by different hardware units.
  • various units may be combined in a codec hardware unit in combination with suitable software and/or firmware, or by interoperating hardware units (including one or more processors as described above) provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了点云编解码方法和编解码器，涉及编解码技术领域，有助于提高编解码性能。点云译码方法(包括点云编码方法或点云解码方法)包括基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型；其中，无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；目标patch占用图是待处理边界像素块所属的patch占用图；根据待处理边界像素块的类型，重构待译码点云。

Description

点云编解码方法和编解码器
本申请要求于2018年12月13日提交国家知识产权局、申请号为201811527977.3、申请名称为“点云编解码方法和编解码器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及编解码技术领域，尤其涉及点云(point cloud)编解码方法和编解码器。
背景技术
随着3d传感器(例如3d扫描仪)技术的不断发展，采集点云数据越来越便捷，所采集的点云数据的规模也越来越大，面对海量的点云数据如何有效存储变为一个很迫切需要解决的问题。对点云的高质量压缩、存储和传输就变得非常重要。
在重构点云的过程中，为了消除由点云占用图的压缩误差带来的重构点云的误差，通常需要对重构的点云占用图的边界像素块进行特殊处理，如填充、滤波、膨胀操作、腐蚀操作等；在这些处理过程中，通常需要根据点云占用图的边界像素块的局部信息决定对其所采用的处理方法，因此需要对这些边界像素块进行分类，而对边界像素块的分类的精确度，直接影响点云占用图的编解码，从而影响点云编解码效果。因此，如何对边界像素块进行分类，从而提高点云编解码效率，成为亟待解决的技术问题。
发明内容
本申请实施例提供了点云编解码方法和编解码器，有助于提高编解码性能。
第一方面，提供了一种点云译码方法，包括：基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型；其中，无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；目标patch占用图是待处理边界像素块所属的patch占用图；根据待处理边界像素块的类型，重构待译码点云。其中，待译码点云是当前帧点云。由于将待压缩点云分成多个patch，再将patch映射成patch占用图，以及对各patch占用图进行打包的点云编解码方法，会导致patch边界附近的重构误差较大。因此，为了保证重构的点云的质量，通常需要对patch占用图的边界处进行特殊处理，比如滤波、补洞等。本申请实施例提供的点云编码方法中，基于待处理边界像素块的空域相邻边界像素块是否无效像素块，确定待处理边界像素块的类型，从而基于该类型重构点云。其中，无效像素块包括所包含的像素的值不全为0且不属于目标patch占用图的像素块。这样，有助于识别出待译码点云的占用图中重构误差较大的边界像素块，从而可以对其采取不同的处理方法，以保证重构点云的质量，因此，可以提高系统编解码性能。
在一种可能的设计中，该方法还包括：当待译码点云的占用图的当前边界像素块的空域相邻像素块中的无效像素块的个数大于或等于预设阈值时，确定当前边界像素 块为待处理边界像素块。这是在考虑到如下因素而提出的技术方案：当待译码点云的占用图的当前边界像素块的空域相邻像素块中的有效像素块的个数大于某一阈值时，当前边界像素块是满块的概率会较大，因此，可以不对该当前边界像素块进行处理。
在一种可能的设计中，基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型包括：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息；其中，不同类型的边界像素块对应不同的方位信息。
在一种可能的设计中，如果待处理边界像素块的预设方位的空域相邻像素块是无效像素块，则确定待处理边界像素块中的无效像素在待处理边界像素块中的预设方位；其中，预设方位是正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方中的其中一种或者至少两种的组合。
在一种可能的设计中，根据待处理边界像素块的类型，重构待译码点云，包括：根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块；并根据经处理过的占用图，重构待译码点云，经处理过的占用图包括经目标操作的像素块；其中，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块，包括：当目标操作是置0操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置0，得到经置0的像素块，不同处理方式对应的目标位置不同；或者，当目标操作是置1操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置1，得到经置1的像素块，不同处理方式对应的目标位置不同；或者，当目标操作是膨胀操作时，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行膨胀操作，得到经膨胀操作后的像素块，不同处理方式的卷积核的半径不同。
需要说明的是，本申请实施例中所描述的置0操作是为了使得一个像素块中的像素没有被占用的操作，置1操作是为了使得一个像素块中的像素没有占用的操作。例如，置0操作的执行结果是一个像素的值是0，置1操作的执行结果是一个像素的值是1，下文中也均以此为例进行说明，在此统一说明，下文不再赘述。也就是说，0和1只是为了区分占有和非占有，实际实现时可以替换成其他值或者一个范围。例如，可以使用“4”表示占有，非4表示非占用等。
在一种可能的设计中，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块，包括：根据边界像素块的多种类型与多种处理方式之间的映射关系，确定待处理边界像素块的类型对应的处理方式；若待处理边界像素块的类型对应一种处理方式，则将待处理边界像素块的类型对应的处理方式作为目标处理方式；或者，若待处理边界像素块的类型对应多种处理方式，则将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式；采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
在一种可能的设计中，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块，包括：根据待处理边 界像素块的类型查表，得到待处理边界像素块的类型对应的处理方式，表包括边界像素块的多种类型与多种处理方式之间的映射关系；若待处理边界像素块的类型对应一种处理方式，则将待处理边界像素块的类型对应的处理方式作为目标处理方式；或者，若待处理边界像素块的类型对应多种处理方式，则将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式；采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
在一种可能的设计中，将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式，包括：根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式；目标空域相邻像素块是待处理边界像素块的为无效像素块的空域相邻像素块。
在一种可能的设计中，根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式，包括：如果目标空域相邻像素块是空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第一处理方式作为目标处理方式；如果目标空域相邻像素块是非空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第二处理方式作为目标处理方式；其中，当目标操作是置0操作时，第一处理方式对应的目标位置的范围大于第二处理方式对应的目标位置的范围；或者，当目标操作是置1操作时，第一处理方式对应的目标位置的范围小于第二处理方式对应的目标位置的范围；或者，当目标操作是膨胀操作时，第一处理方式对应的卷积核的半径小于第二处理方式对应的卷积核的半径。这有助于提高编解码效率。
在一种可能的设计中，待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻且位于待处理边界像素块的正上方、正下方、正左方和正右方的像素块；若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括正上方、正下方、正左方和正右方中的其中一种或至少两种的组合；或者，若待处理边界像素块的正上方和正右方的像素块为无效像素块，且待处理边界像素块的正下方和正左方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的右上方；或者，若待处理边界像素块的正下方和正左方的像素块为无效像素块，且待处理边界像素块的正上方和正右方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左下方；或者，若待处理边界像素块的正上方和正左方的像素块为无效像素块，且待处理边界像素块的正下方和正右方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左上方；或者，若待处理边界像素块的正下方和正右方的像素块为无效像素块，且待处理边界像素块的正上方和正左方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的右下方。
在一种可能的设计中，待处理边界像素块的空域相邻像素块包括与待处理边界像素块相邻的且位于待处理边界像素块的左上方、右上方、左下方和右下方的像素块； 若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方和右下方其中一种或至少两种。
在一种可能的设计中，待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻的且位于待处理边界像素块的正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方的像素块；若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方或右下方。
在一种可能的设计中，当目标操作是置0操作时，目标位置是待处理边界像素块中的，与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置；或者，目标位置是待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置；直线与待处理边界像素块的类型相关；
在一种可能的设计中，当目标操作是置1操作时，目标位置是待处理边界像素块中的，与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置；或者，目标位置是待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置；直线与待处理边界像素块的类型相关。
在一种可能的设计中，基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型包括：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息；基于待处理边界像素块的为无效像素块的空域相邻像素块是否是空无效像素块，以及确定的方位信息，确定待处理边界像素块的类型。
在一种可能的设计中，待译码点云的占用图包括第一待处理边界像素块和第二待处理边界像素块；第一待处理边界像素块的类型是第一类型，第二待处理边界像素块的类型是第二类型；第一类型与第二类型所对应的方位信息相同；且第一待处理边界像素块的空域相邻像素块中预设方位的无效像素块是空无效像素块，第二待处理边界像素块的空域相邻像素块中预设方位的无效像素块是非空无效像素块。
在一种可能的设计中，待译码点云是待编码点云，若待处理边界像素块的类型对应多种处理方式；方法还包括：将标识信息编入码流，标识信息表示待处理边界像素块的目标处理方式。
在一种可能的设计中，待译码点云是待解码点云，若待处理边界像素块的类型对应多种处理方式，采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块，包括：根据待处理边界像素块的类型，解析码流，以得到标识信息；标识信息表示目标处理方式；采用标识信息所指示的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
在一种可能的设计中，待译码点云是待解码点云，方法还包括：解析码流，以得到待译码点云的待处理边界像素块的尺寸信息；根据尺寸信息对待解码点云的占用图进行划分，得到一个或多个待处理边界像素块。
第二方面，提供了一种点云编码方法，包括：确定指示信息，该指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；目标编码方法包括如第一方面或第一方面的任一种可能的设计提供的点云译码方法(具体是点云编码方法)；将该指示信息编入码流。
第三方面，提供了一种点云解码方法，包括：解析码流，以得到指示信息，该指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；目标解码方法包括第一方面或第一方面的任一种可能的设计提供的点云译码方法(具体是点云解码方法)；当该指示信息用于指示按照目标解码方法对待解码点云的占用图进行处理时，按照目标解码方法对待解码点云的占用图进行处理。
第四方面，提供了一种译码器，包括：分类模块，基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型；其中，所述无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，所述无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；所述目标patch占用图是所述待处理边界像素块所属的patch占用图，重构模块，根据所述待处理边界像素块的类型，重构所述待译码点云。例如，分类模块可以对应图2中的占用图滤波模块112中的分类子模块。重构模块可以对应图2中的占用图滤波模块112中的预处理子模块，以及点云重构模块111。
第五方面，提供了一种编码器，包括：辅助信息编码模块，用于确定指示信息，以及将该指示信息编入码流；该指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；目标编码方法包括上述第一方面或第一方面的任一种可能的设计提供的点云译码方法(具体是点云编码方法)。
第六方面，提供了一种解码器，包括：辅助信息解码模块，用于解析码流，以得到指示信息，该指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；目标解码方法包括上述第一方面或第一方面的任一种可能的设计提供的点云译码方法(具体是点云解码方法)。占用图滤波模块，用于当该指示信息用于指示按照目标解码方法对待解码点云的占用图进行处理时，按照目标解码方法对待解码点云的占用图进行处理。
第七方面，提供一种译码装置，包括：存储器和处理器；其中，该存储器用于存储程序代码；该处理器用于调用该程序代码，以执行上述第一方面或第一方面的任一种可能的设计提供的点云译码方法。
第八方面，提供一种编码装置，包括：存储器和处理器；其中，该存储器用于存储程序代码；该处理器用于调用该程序代码，以执行上述第二方面提供的点云编码方法。
第九方面，提供一种解码装置，包括：存储器和处理器；其中，该存储器用于存储程序代码；该处理器用于调用该程序代码，以执行上述第三方面提供的点云编码方法。
本申请还提供一种计算机可读存储介质，包括程序代码，该程序代码在计算机上运行时，使得该计算机执行如上述第一方面及其可能的设计提供的任一种点云译码方 法。
本申请还提供一种计算机可读存储介质，包括程序代码，该程序代码在计算机上运行时，使得该计算机执行上述第二方面提供的点云编码方法。
本申请还提供一种计算机可读存储介质，包括程序代码，该程序代码在计算机上运行时，使得该计算机执行上述第三方面提供的点云编码方法。
应当理解的是，上述提供的任一种编解码器、处理装置、编解码装置和计算机可读存储介质的有益效果均可以对应参考上文对应方面提供的方法实施例的有益效果，不再赘述。
附图说明
图1为可用于本申请实施例的一种实例的点云译码系统的示意性框图；
图2为可用于本申请实施例的一种实例的编码器的示意性框图；
图3为可适用于本申请实施例的一种点云、点云的patch以及点云的占用图的示意图；
图4为可用于本申请实施例的一种实例的解码器的示意性框图；
图5为本申请实施例提供的一种点云译码方法的流程示意图；
图6为本申请实施例提供的一种目标位置的示意图；
图7为本申请实施例提供的另一种目标位置的示意图；
图8为本申请实施例提供的另一种目标位置的示意图；
图9为本申请实施例提供的像素块的类型的索引、判别方式图、示意图以及描述信息的对应关系的示意图；
图10为本申请实施例提供的一种确定目标位置的像素的示意图；
图11为本申请实施例提供的另一种确定目标位置的像素的示意图；
图12为本申请实施例提供的另一种确定目标位置的像素的示意图；
图13为本申请实施例提供的另一种确定目标位置的像素的示意图；
图14为本申请实施例提供的另一种确定目标位置的像素的示意图；
图15为本申请实施例提供的另一种确定目标位置的像素的示意图；
图16为本申请实施例提供的另一种确定目标位置的像素的示意图；
图17为可适用于本申请一实施例的几种卷积核的示意图；
图18为本申请实施例提供的两种类型为1的待处理边界像素块在填充之前的像素块的示意图；
图19为本申请实施例提供的一种码流结构的示意图；
图20为本申请实施例提供的一种点云编码方法的流程示意图；
图21为本申请实施例提供的一种点云解码方法的流程示意图；
图22为本申请实施例提供的一种译码器的示意性框图；
图23为本申请实施例提供的另一种译码器的示意性框图；
图24为本申请实施例提供的另一种译码器的示意性框图；
图25为本申请实施例提供的一种编码器的示意性框图；
图26为本申请实施例提供的一种解码器的示意性框图；
图27为用于本申请实施例的译码设备的一种实现方式的示意性框图。
具体实施方式
本申请实施例中的术语“至少一个(种)”包括一个(种)或多个(种)。“多个(种)”是指两个(种)或两个(种)以上。例如，A、B和C中的至少一种，包括：单独存在A、单独存在B、同时存在A和B、同时存在A和C、同时存在B和C，以及同时存在A、B和C。在本申请的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“多个”是指两个或多于两个。另外，为了便于清楚描述本申请实施例的技术方案，在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
图1为可用于本申请实施例的一种实例的点云译码系统1的示意性框图。术语“点云译码”或“译码”可一般地指代点云编码或点云解码。点云译码系统1的编码器100可以根据本申请提出的任一种点云编码方法对待编码点云进行编码。点云译码系统1的解码器200可以根据本申请提出的与编码器使用的点云编码方法相对应的点云解码方法对待解码点云进行解码。
如图1所示，点云译码系统1包含源装置10和目的地装置20。源装置10产生经编码点云数据。因此，源装置10可被称为点云编码装置。目的地装置20可对由源装置10所产生的经编码的点云数据进行解码。因此，目的地装置20可被称为点云解码装置。源装置10、目的地装置20或两个的各种实施方案可包含一或多个处理器以及耦合到所述一或多个处理器的存储器。所述存储器可包含但不限于随机存取存储器(random access memory，RAM)、只读存储器(read-only memory，ROM)、带电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、快闪存储器或可用于以可由计算机存取的指令或数据结构的形式存储所要的程序代码的任何其它媒体，如本文所描述。
源装置10和目的地装置20可以包括各种装置，包含桌上型计算机、移动计算装置、笔记型(例如，膝上型)计算机、平板计算机、机顶盒、例如所谓的“智能”电话等电话手持机、电视机、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机或其类似者。
目的地装置20可经由链路30从源装置10接收经编码点云数据。链路30可包括能够将经编码点云数据从源装置10移动到目的地装置20的一或多个媒体或装置。在一个实例中，链路30可包括使得源装置10能够实时将经编码点云数据直接发送到目的地装置20的一或多个通信媒体。在此实例中，源装置10可根据通信标准(例如无线通信协议)来调制经编码点云数据，且可将经调制的点云数据发送到目的地装置20。所述一或多个通信媒体可包含无线和/或有线通信媒体，例如射频(radio frequency，RF)频谱或一或多个物理传输线。所述一或多个通信媒体可形成基于分组的网络的一部分，基于分组的网络例如为局域网、广域网或全球网络(例如，因特网)。所述一或多个通信媒体可包含路由器、交换器、基站或促进从源装置10到目的地装置20的通信的其它设备。
在另一实例中，可将经编码数据从输出接口140输出到存储装置40。类似地，可通过输入接口240从存储装置40存取经编码点云数据。存储装置40可包含多种分布式或本地存取的数据存储媒体中的任一者，例如硬盘驱动器、蓝光光盘、数字多功能光盘(digital versatile disc，DVD)、只读光盘(compact disc read-only memory，CD-ROM)、快闪存储器、易失性或非易失性存储器，或用于存储经编码点云数据的任何其它合适的数字存储媒体。
在另一实例中，存储装置40可对应于文件服务器或可保持由源装置10产生的经编码点云数据的另一中间存储装置。目的地装置20可经由流式传输或下载从存储装置40存取所存储的点云数据。文件服务器可为任何类型的能够存储经编码的点云数据并且将经编码的点云数据发送到目的地装置20的服务器。实例文件服务器包含网络服务器(例如，用于网站)、文件传输协议(file transfer protocol，FTP)服务器、网络附属存储(network attached storage，NAS)装置或本地磁盘驱动器。目的地装置20可通过任何标准数据连接(包含因特网连接)来存取经编码点云数据。这可包含无线信道(例如，Wi-Fi连接)、有线连接(例如，数字用户线路(digital subscriber line，DSL)、电缆调制解调器等)，或适合于存取存储在文件服务器上的经编码点云数据的两者的组合。经编码点云数据从存储装置40的传输可为流式传输、下载传输或两者的组合。
图1中所说明的点云译码系统1仅为实例，并且本申请的技术可适用于未必包含点云编码装置与点云解码装置之间的任何数据通信的点云译码(例如，点云编码或点云解码)装置。在其它实例中，数据从本地存储器检索、在网络上流式传输等等。点云编码装置可对数据进行编码并且将数据存储到存储器，和/或点云解码装置可从存储器检索数据并且对数据进行解码。在许多实例中，由并不彼此通信而是仅编码数据到存储器和/或从存储器检索数据且解码数据的装置执行编码和解码。
在图1的实例中，源装置10包含数据源120、编码器100和输出接口140。在一些实例中，输出接口140可包含调节器/解调器(调制解调器)和/或发送器(或称为发射器)。数据源120可包括点云捕获装置(例如，摄像机)、含有先前捕获的点云数据的点云存档、用以从点云内容提供者接收点云数据的点云馈入接口，和/或用于产生点云数据的计算机图形系统，或点云数据的这些来源的组合。
编码器100可对来自数据源120的点云数据进行编码。在一些实例中，源装置10经由输出接口140将经编码点云数据直接发送到目的地装置20。在其它实例中，经编码点云数据还可存储到存储装置40上，供目的地装置20以后存取来用于解码和/或播放。
在图1的实例中，目的地装置20包含输入接口240、解码器200和显示装置220。在一些实例中，输入接口240包含接收器和/或调制解调器。输入接口240可经由链路30和/或从存储装置40接收经编码点云数据。显示装置220可与目的地装置20集成或可在目的地装置20外部。一般来说，显示装置220显示经解码点云数据。显示装置220可包括多种显示装置，例如，液晶显示器(liquid crystal display，LCD)、等离子显示器、有机发光二极管(organic light-emitting diode，OLED)显示器或其它类型的显示装置。
尽管图1中未图示，但在一些方面，编码器100和解码器200可各自与音频编码 器和解码器集成，且可包含适当的多路复用器-多路分用器(multiplexer-demultiplexer，MUX-DEMUX)单元或其它硬件和软件，以处置共同数据流或单独数据流中的音频和视频两者的编码。在一些实例中，如果适用的话，那么MUX-DEMUX单元可符合ITU H.223多路复用器协议，或例如用户数据报协议(user datagram protocol，UDP)等其它协议。
编码器100和解码器200各自可实施为例如以下各项的多种电路中的任一者：一或多个微处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)、离散逻辑、硬件或其任何组合。如果部分地以软件来实施本申请，那么装置可将用于软件的指令存储在合适的非易失性计算机可读存储媒体中，且可使用一或多个处理器在硬件中执行所述指令从而实施本申请技术。前述内容(包含硬件、软件、硬件与软件的组合等)中的任一者可被视为一或多个处理器。编码器100和解码器200中的每一者可包含在一或多个编码器或解码器中，所述编码器或解码器中的任一者可集成为相应装置中的组合编码器/解码器(编码解码器)的一部分。
本申请可大体上将编码器100称为将某些信息“发信号通知”或“发送”到例如解码器200的另一装置。术语“发信号通知”或“发送”可大体上指代用以对经压缩点云数据进行解码的语法元素和/或其它数据的传送。此传送可实时或几乎实时地发生。替代地，此通信可经过一段时间后发生，例如可在编码时在经编码位流中将语法元素存储到计算机可读存储媒体时发生，解码装置接着可在所述语法元素存储到此媒体之后的任何时间检索所述语法元素。
如图2所示，为可用于本申请实施例的一种实例的编码器100的示意性框图。图2是以MPEG(Moving Picture Expert Group)点云压缩(Point Cloud Compression，PCC)编码框架为例进行说明的。在图2的实例中，编码器100可以包括patch信息生成模块101、打包模块102、深度图生成模块103、纹理图生成模块104、第一填充模块105、基于图像或视频的编码模块106、占用图编码模块107、辅助信息编码模块108和复用模块109等。另外，编码器100还可以包括点云滤波模块110、点云重构模块111和占用图滤波模块112等。其中：
patch信息生成模块101，用于采用某种方法将一帧点云分割产生多个patch，以及获得所生成的patch的相关信息等。其中，patch是指一帧点云中部分点构成的集合，通常一个连通区域对应一个patch。patch的相关信息可以包括但不限于以下信息中的至少一项：点云所分成的patch的个数、每个patch在三维空间中的位置信息、每个patch的法线坐标轴的索引、每个patch从三维空间投影到二维空间产生的深度图、每个patch的深度图大小(例如深度图的宽和高)、每个patch从三维空间投影到二维空间产生的占用图等。该相关信息中的部分，如点云所分成的patch的个数，每个patch的法线坐标轴的索引，每个patch的深度图大小、每个patch在点云中的位置信息、每个patch的占用图的尺寸信息等，可以作为辅助信息被发送到辅助信息编码模块108，以进行编码(即压缩编码)。每个patch的占用图可以被发送到打包模块102进行打包，具体的，将该点云的各patch按照特定的顺序进行排列例如按照各patch的占用图的宽/高降序(或升序)排列；然后，按照排列后的各patch的顺序，依次将patch的占用图 插入该点云占用图的可用区域中，得到该点云的占用图。再一方面，各patch在该点云占用图中的具体位置信息和各patch的深度图等可以被发送到深度图生成模块103。
[根据细则91更正 30.12.2019]　
打包模块102获得该点云的占用图后，一方面可以将该点云的占用图发送到占用图编码模块107以进行编码。另一方面可以利用该点云的占用图指导深度图生成模块103生成该点云的深度图和指导纹理图生成模块104生成该点云的纹理图。
如图3所示，为可适用于本申请实施例的一种点云、点云的patch以及点云的占用图的示意图。其中，图3中的(a)图为一帧点云的示意图，图3中的(b)图为基于图3中的(a)图获得的点云的patch的示意图，图3中的(c)图为图3中的(b)图所示的各patch映射到二维平面上所得到的各patch的占用图经打包得到的该点云的占用图的示意图。
深度图生成模块103，用于根据该点云的占用图、该点云的各patch的占用图和深度信息，生成该点云的深度图，并将所生成的深度图发送到第一填充模块105，以对深度图中的空白像素点进行填充，得到经填充的深度图。
纹理图生成模块104，用于根据该点云的占用图、该点云的各patch的占用图和纹理信息，生成该点云的纹理图，并将所生成的纹理图发送到第一填充模块105，以对纹理图中的空白像素点进行填充，得到经填充的纹理图。
经填充的深度图和经填充的纹理图被第一填充模块105发送到基于图像或视频的编码模块106，以进行基于图像或视频的编码。后续：
一方面，基于图像或视频的编码模块106、占用图编码模块107、辅助信息编码模块108，将所得到的编码结果(即码流)发送到复用模块109，以合并成一个码流，该码流可以被发送到输出接口140。
另一方面，基于图像或视频的编码模块106所得到的编码结果(即码流)发送到点云重构模块111进行点云重构得到经重构的点云(具体是得到重构的点云几何信息)。具体的，对基于图像或视频的编码模块106所得到的经编码的深度图进行视频解码，获得该点云的解码深度图，利用解码深度图、该点云的占用图和各patch的辅助信息，获得重构的点云几何信息。其中，点云的几何信息是指点云中的点(例如点云中的每个点)在三维空间中的坐标值。应用于在本申请实施例时，这里的“该点云的占用图”可以是该点云经占用图滤波模块112滤波(或称为平滑处理)后得到的占用图。可选的，点云重构模块111还可以将该点云的纹理信息和重构的点云几何信息发送到着色模块，着色模块用于对重构点云进行着色，以获得重构点云的纹理信息。可选的，纹理图生成模块104还可以基于经点云滤波模块110对重构的点云几何信息进行滤波得到的信息生成该点云的纹理图。
以下，对占用图滤波模块112进行详细描述。
占用图滤波模块112可以位于打包模块102和点云重构模块111，用于对经填充的点云的占用图进行滤波。
在一个示例中，编码器100还包括第二填充单元，用于对占用图滤波模块112生成的点云的占用图进行填充。在另一个示例中，可以认为由占用图滤波模块112执行对来自打包模块102的点云占用图进行填充，为了描述方便，本申请实施例中均以该 示例为例进行说明。
在一个示例中，占用图滤波模块112可以包括分类子模块、预处理子模块和滤波子模块。当然，还可以包括填充子模块，用于对占用图滤波模块112生成的点云的占用图进行填充。其中，分类子模块用于对当前帧点云的占用图的边界像素块进行分类，预处理子模块用于基于分类结果对当前帧点云的占用图进行处理(如置0操作或置1操作或膨胀操作等)，滤波子模块，用于对经处理的点云的占用图进行滤波(也可以称为平滑处理)。
可选的，占用图滤波模块112还与打包模块102和辅助信息编码模块108连接，如图2中的虚线所示。占用图滤波模块112还用于根据打包模块102发送的点云的占用图确定待处理边界像素块对应的目标处理方式，并将目标处理方式的标识信息作为辅助信息发送给辅助信息编码模块108，并由辅助信息编码模块108将该标识信息编入码流。
需要说明的是，该可选的实现方式中将目标处理方式的标识信息作为辅助信息并由辅助信息编码模块108将其编入码流为例进行说明的，可替换的，目标处理方式的标识信息也可以由独立于辅助信息编码模块108的一个编码模块将其编入码流，并将该码流发送到复用模块109，以得到合并的码流。另外，该可选的实现方式中是以占用图滤波模块112根据打包模块102发送的点云的占用图确定待处理边界像素块对应的目标处理方式为例进行说明的，可替换的，占用图滤波模块112也可以不依赖于打包模块102发送的点云的占用图，确定目标处理方式。该情况下，占用图滤波模块112可以不与打包模块102连接。
可选的，占用图滤波模块112还可以与patch信息生成模块101连接，用于从patch信息生成模块101中获取辅助信息如Block2PatchIndex等。
可以理解的，图2所示的编码器100仅为示例，具体实现时，编码器100可以包括比图2中所示的更多或更少的模块。本申请实施例对此不进行限定。
如图4所示，为可用于本申请实施例的一种实例的解码器200的示意性框图。其中，图4中是以MPEG PCC解码框架为例进行说明的。在图4的实例中，解码器200可以包括解复用模块201、基于图像或视频的解码模块202、占用图解码模块203、辅助信息解码模块204、点云重构模块205、点云滤波模块206和点云的纹理信息重构模块207。另外，解码器200可以包括占用图滤波模块208。其中：
解复用模块201用于将输入的码流(即合并的码流)发送到相应解码模块。具体的，将包含经编码的纹理图的码流和经编码的深度图的码流发送给基于图像或视频的解码模块202；将包含经编码的占用图的码流发送给占用图解码模块203，将包含经编码的辅助信息的码流发送给辅助信息解码模块204。
基于图像或视频的解码模块202，用于对接收到的经编码的纹理图和经编码的深度图进行解码；然后，将解码得到的纹理图信息发送给点云的纹理信息重构模块207，将解码得到的深度图信息发送给点云重构模块205。占用图解码模块203，用于对接收到的包含经编码的占用图的码流进行解码，并将解码得到的占用图信息发送给点云重构模块205。应用于在本申请实施例时，发送给点云重构模块205的占用图信息可以是经占用图滤波模块208进行滤波后得到的占用图的信息。辅助信息解码模块204， 用于对接收到的经编码的辅助信息进行解码，并将解码得到的指示辅助信息的信息发送给点云重构模块205。
点云重构模块205，用于根据接收到的占用图信息和辅助信息对点云的几何信息进行重构，具体的重构过程可以参考编码器100中的点云重构模块111的重构过程，此处不再赘述。经重构的点云的几何信息经点云滤波模块206滤波之后，被发送到点云的纹理信息重构模块207。点云的纹理信息重构模块207用于对点云的纹理信息进行重构，得到经重构的点云。
以下，对占用图滤波模块208进行详细描述。
占用图滤波模块208位于占用图解码模块203与点云重构模块205之间，用于对占用图解码模块203发送的占用图信息所表示的占用图进行滤波，并将滤波得到的占用图的信息发送给点云重构模块205。其中，这里的占用图是点云经填充的占用图。
在一个示例中，占用图滤波模块208可以包括分类子模块、预处理子模块和滤波子模块。各子模块所执行的步骤可以参考上文中对占用图滤波模块112中相应子模块的描述。
可选的，占用图滤波模块208还与辅助信息解码模块204连接，如图4中的虚线所示。例如，占用图滤波模块208可以用于接收辅助信息解码模块204解析码流得到的目标处理方式的标识信息。又如，占用图滤波模块208还可以用于接收辅助信息解码模块204解析码流得到的Block2PatchIndex等。
可以理解的，图4所示的解码器200仅为示例，具体实现时，解码器200可以包括比图4中所示的更多或更少的模块。本申请实施例对此不进行限定。
需要说明的是，编码器100中的点云滤波模块110，以及解码器200中的点云滤波模块206可以去除经重构的点云中的明显的噪声特征的像素如游离点、毛躁边界等。也就是说，使用点云滤波模块能够去除经重构的点云中的一部分outlier点(即离群点或异常点)。但是，如果仅通过点云滤波模块能够去除经重构的点云中的outlier点，则效果不佳。
考虑到经重构的点云中的outlier点的根本原因是由于对点云的占用图进行填充，本申请实施例提供了新的点云编解码方法和编解码器。
为了便于理解本申请实施例提供的技术方案，以下，对填充过程进行说明。
填充是为了节省码流开销而引入的对点云的占用图进行处理的步骤。一种填充方法可以包括：遍历点云的占用图中的每个B0*B0的像素块，不同B0*B0的像素块之间无交叠。对于任意一个B0*B0的像素块来说，如果该像素块中的至少一个像素的值为1，则将该像素块内的所有像素的值均填充为1(即均置为1)。其中，B0*B0是执行填充的基本填充单位。B0是一个基本填充单位包含的像素的一行/列包含的像素的个数。B0通常取2个整数幂，例如B0＝2、4、8、16。填充后的点云的占用图的分辨率是B0*B0，在此统一说明，下文不再赘述。该填充方法仅为示例，其不对本申请实施例可适用的填充方法构成限定。
虽然执行填充操作可以节省码流开销，但是，会导致如下问题：
第一，会导致点云经填充的占用图产生锯齿状的边缘。被填充的像素(即填充之前像素值为0且填充之后像素值为1的像素)在重构之后，会成为经重构的点云中的 outlier点。
第二，由于对深度图的编码为有损编码(如：H.265编码器)，量化误差会导致在一定概率上使得点云中的两个点出现在同一个位置，若量化误差越大，点云上的两个点出现在同一个位置的概率就越高，进而使得在重建点云在patch的边界上出现空洞。
为此，本申请实施例提供了能够有效减少点云的占用图因填充导致重构后的点云出现的outlier点，从而提高编解码性能。具体的，在编码端和/或解码端重构点云之前，对该点云经填充的占用图进行滤波，并采用该点云滤波后的占用图重构点云。
针对此，一种解决方案，在重构点云之前，基于点云的占用图中的边界像素块的类型对该边界像素块进行处理，如置0操作、置1操作或膨胀操作等。因此，如何确定点云的占用图中的边界像素块的类型会直接影响处理结果与真实的(即未填充前的)点云的占用图之间的差异，从而影响编解码性能。
当然，本申请实施例提供的点云译码方法中的确定点云的占用图中的边界像素块的类型的方法也可以应用于其他场景中。
以下，对本申请实施例提供的点云编码、解码方法进行说明。需要说明的是，结合图1所示的点云译码系统，下文中的任一种点云编码方法可以是点云译码系统中的源装置10执行的，更具体的，是由源装置10中的编码器100执行的；下文中的任一种点云解码方法可以是点云译码系统中的目的装置20执行的，更具体的，是由目的装置20中的解码器200执行的。
为了描述上的简洁，如果不加说明，下文中描述的点云译码方法可以包括点云编码方法或点云解码方法。当点云译码方法具体是点云编码方法时，图5所示的实施例中的待译码点云具体是待编码点云；当点云译码方法具体是点云解码方法时，图5所示的实施例中的待译码点云具体是待解码点云。
如图5所示，为本申请实施例提供的一种点云译码方法的流程示意图。该方法可以包括：
S101：基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型。其中，无效像素块包括所包含的像素的值不全为0且不属于目标patch占用图的像素块；或者，无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；目标patch占用图是待处理边界像素块所属的patch占用图。
本申请实施例中的待译码点云的占用图可以是指待译码点云填充前的占用图，也可以是指待译码点云填充后的占用图。
待译码点云的占用图中的像素块分为有效像素块和无效像素块。进一步地，无效像素块分为空无效像素块和非空无效像素块。其中，空无效像素块是所包含的像素的值全为0的像素块。非空无效像素块是所包含的像素的值不全为0且不属于目标patch占用图的像素块。目标patch占用图是待处理边界像素块所属的patch占用图。相应的，将所包含的像素的值不全为0且属于目标patch占用图的像素块作为有效像素块。
可以理解的是，本申请实施例中的非空无效像素块是相对待处理边界像素块而言的，因此，待译码点云的占用图中可能存在相对待处理边界像素块1来说，某个像素块是无效像素块；相对待处理边界像素块2来说，该像素块是有效像素块的情况。
一个像素块的空域相邻像素块包括与该像素块相邻的，且位于该像素块的正上方、正下方、正左方、正右方、左上方、左下方、右上方和右下方的一个或多个像素块。具体实现的过程中，译码器可以根据两个像素块的坐标，确定这两个像素块是否相邻，以及这两个像素块中的一个像素块相对另一个像素块的方位。
如果待译码点云的占用图中的一个像素块的所有空域相邻像素块均为有效像素块，则该待处理像素块为待译码点云的占用图的非边界像素块。否则，该像素块是待译码点云的占用图的边界像素块。
在本申请的一些实施例中，待处理边界像素块可以是待译码点云的占用图中的任意一个边界像素块。
在本申请的另一些实施例中，待处理边界像素块可以是待译码点云的占用图中的满足某一条件的边界像素块，例如，当待译码点云的占用图的当前边界像素块(即任意一个边界像素块)的空域相邻像素块中的无效像素块的个数大于或等于预设阈值(即有效像素块的个数小于或等于阈值)时，确定该当前边界像素块为待处理边界像素块。例如，预设阈值可以是6，当然本申请实施例不限于此。这是在考虑到如下因素而提出的技术方案：当待译码点云的占用图的当前边界像素块的空域相邻像素块中的有效像素块的个数大于某一阈值时，当前边界像素块是满块的概率会较大，因此，可以不对该当前边界像素块进行处理。
可选的，待处理边界像素块是对待译码点云的占用图执行填充的基本填充单位，当然本申请实施例不限于此。
本申请实施例对如何确定一个像素块所属的patch占用图不进行限定。例如，可以参考现有技术。另外，本申请实施例提供了如下技术方案：
基于待译码点云的辅助信息中的Block2PatchIndex，获得像素块所在的编码块与patch之间的对应关系，从而根据该对应关系，确定该像素块所属的patch占用图。
其中，编码块是打包的基本单位。通常，一个编码块大于或等于像素块，例如，编码块通常大于或等于像素块，且编码块的尺寸通常是像素块的尺寸的整数倍。比如，编码块的大小是16x16，像素块的大小是4x4。一个patch占用图会占用多个编码块，而一个编码块只能被一个patch所占用。尽管如此，多个patch的包围盒(boundingbox)会有重叠，因此需要用Block2PatchIndex来指明每个编码块是被包围盒在该编码块处有重叠的多个patch中的哪一个patch所占用的信息，如记录编码块的编号与patch的编号之间的对应关系。其中，patch的编号与patch占用图的编号是相对应的。因此，基于Block2PatchIndex可以获得编码块所在的patch占用图的编号，从而得到相关的像素块所在的patch占用图的编号。由于辅助信息一般是无损编码的，所以编码器和解码器得到的Block2PatchIndex是一样的。
在一种实现方式中，译码器可以以像素块为单位执行S101。具体的：对于第i个像素块来说，首先根据Block2PatchIndex确定待处理边界像素块所在的编码块对应的patch占用图，并将该patch占用图作为目标patch占用图，以及待处理边界像素块的非空空域相邻像素块所在的编码块对应的patch占用图，从而确定待处理像素块是否是待处理边界像素块；接着根据目标patch占用图，确定待处理边界像素块的类型。其中，1≤i≤I，i和I均是整数。I是待译码点云的占用图中的像素块的个数。其中， 该实现方式中的第i个像素块可以替换为第i个包含非0值的像素的像素块；相应地，I的含义可以替换为是待译码点云的占用图中的包含非0值的像素的像素块的个数。
在另一种实现方式中，译码器可以以patch占用图(或patch)为单位执行S101。具体的：对于第j个patch占用图来说，基于Block2PatchIndex确定其所占的编码块，并依次确定每个编码块中的像素块(具体可以是指包含的非0值的像素的像素块)是否是待处理边界像素块，并根据第j个patch占用图(即目标patch占用图)，确定待处理边界像素块的类型。其中，1≤j≤J，j和J均是整数。j是待译码点云的占用图中的patch占用图的个数，即待译码点云包括的patch的个数。
S102：根据待处理边界像素块的类型，重构待译码点云。具体实现方式可以参考下文。
由于将待压缩点云分成多个patch，再将patch映射成patch占用图，以及对各patch占用图进行打包的点云编解码方法，会导致patch边界附近的重构误差较大。因此，为了保证重构的点云的质量，通常需要对patch占用图的边界处进行特殊处理，比如滤波、补洞等。本申请实施例提供的点云编码方法中，基于待处理边界像素块的空域相邻边界像素块是否无效像素块，确定待处理边界像素块的类型，从而基于该类型重构点云。其中，无效像素块包括所包含的像素的值不全为0且不属于目标patch占用图的像素块。这样，有助于识别出待译码点云的占用图中重构误差较大的边界像素块，从而可以对其采取不同的处理方法，以保证重构点云的质量，因此，可以提高系统编解码性能。
可选的，上述S101可以包括以下实现方式：
方式A：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素(或有效像素)在待处理边界像素块中的方位信息；其中，不同类型的待处理边界像素块对应不同的方位信息。例如，先在该点云的经填充的占用图中，获取待处理边界像素块的空域相邻像素块，然后，通过确定这些空域相邻像素块是否是无效像素块(或者是否是有效像素块)，确定待处理边界像素块的类型。
或者，如果待译码点云的占用图是填充后的占用图，则S101可以包括：基于待处理边界像素块在填充前的像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息。例如，先在该点云的填充前的占用图中获取待处理边界像素块在填充前的像素块，以及待处理边界像素块在填充前的像素块的空域相邻像素块，然后，通过确定这些空域相邻像素块是否是无效像素块(或者是否是有效像素块)，确定待处理边界像素块的类型。
待处理边界像素块中的无效像素在待处理边界像素块中的方位信息可以包括以下至少一种：正上方、正下方、正左方、正右方、左上方、左下方、右上方和右下方。可以理解的，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是正上方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是正下方；若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是右上方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是左下方。其他示例与此类似，此处不再一一列举。
需要说明的是，如果不加说明，本申请中的方位信息均是指待处理边界像素块中 的无效像素在待处理边界像素块中的方位信息，在此统一说明，下文不再赘述。
不同类型的待处理边界像素块对应不同的方位信息。例如，若待处理边界像素块中的无效像素在待处理边界像素块的正上方，则可以将待处理边界像素块的类型标记为类型A。再如，若待处理边界像素块中的无效像素在待处理边界像素块的正上方和正下方，则可以将待处理边界像素块的类型标记为类型B。又如，若待处理边界像素块中的无效像素在待处理边界像素块的正上方、正左方和右下方，则可以将待处理边界像素块的类型标记为类型C。其他示例不再一一列举。
可选的，如果待处理边界像素块(或者待处理边界像素块填充之前的像素块)的预设方位的空域相邻像素块是无效像素块，则确定得到待处理边界像素块中的无效像素在待处理边界像素块中的预设方位。其中，该预设方位是正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方中的其中一种或者至少两种的组合。
可以理解的，如果待处理边界像素块的预设方位的像素块是无效像素块，说明待处理边界像素块内部的该预设方位的像素是无效像素的概率，大于该预设方位的像素是有效像素的概率，因此本申请实施例中译码器确定的该预设方位的像素是无效像素。
方式B：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息；基于待处理边界像素块的为无效像素块的空域相邻像素块是否是空无效像素块，以及所确定的方位信息，确定待处理边界像素块的类型。
基于该方式，可能存在如下情况：待译码点云的占用图包括第一待处理边界像素块和第二待处理边界像素块；第一待处理边界像素块的类型是第一类型，第二待处理边界像素块的类型是第二类型；第一类型与第二类型所对应的方位信息相同；且第一待处理边界像素块的空域相邻像素块中预设方位(如至少一个预设方位)的无效像素块是空无效像素块，第二待处理边界图像的空域相邻像素块中该预设方位的无效像素块是非空无效像素块。
也就是说，可能存在边界像素块的不同类型对应同一种方位信息的情况。其中，该不同类型边界像素块的区别在于，各自的空域相邻像素块中相同预设方位的无效像素块的种类不同。可见，方式B提供的技术方案相对方式A来说，基于更细粒度对点云占用图中的边界像素块进行分类。因此，在重构点云的过程中，对边界像素块进行处理的方式可以有更多的选择，从而有助于提高编解码性能。
可以理解的是，在没有冲突的情况下，该方式B中的相关内容的解释可以参考上文。例如，方位信息等的解释。
可选的，上述S102可以包括以下步骤S102A～S102B：
S102A：根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。其中，目标操作可以包括置0操作、置1操作或膨胀操作。膨胀操作可以是计算机视觉中的膨胀操作。可选的，膨胀操作的基本单位小于或等于对待译码点云占用图进行像素值置1的基本单位。
S102B：根据经处理过的占用图，重构待译码点云，经处理过的占用图包括经目标操作的像素块。例如，根据经编码的深度图进行视频解码，获得该点云的解码深度图，利用解码深度图、该点云的经处理过的占用图和各patch的辅助信息，获得重构 的点云几何信息。
从目标操作的具体实现方式不同的角度来看，上述S102A可以通过以下方式之一实现：
方式1：当目标操作是置0操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置0，得到经置0的像素块不同处理方式对应的目标位置不同。
该方式1可以适用于待译码点云的占用图是经填充的占用图的场景中。该方式中，译码器在重构待译码点云之前执行了对待译码点云经填充的占用图的滤波(或平滑处理)。这样，通过合理设置目标位置，有助于将经填充的占用图中的像素值为1的无效像素置0，相比直接采用经填充的占用图重构待译码点云的方案，该方式提供的技术方案获得的经重构的点云中的outlier点较少，因此，有助于提高编解码性能。
方式2：当目标操作是置1操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置1，得到经置1的像素块。不同处理方式对应的目标位置不同。
方式3：当目标操作是膨胀操作时，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行膨胀操作，得到经膨胀操作后的像素块。不同处理方式的卷积核的半径不同。
方式2和方式3可适用于待译码点云的占用图是填充前的占用图的场景中。通过合理设置目标位置或有条件地执行膨胀操作，有助于将点云的占用图中无效像素的值置为1，增加一部分outlier点，在点云平滑时，增加的outlier点可以一定尺度被滤掉，同时又可以将重建点云在patch的边界上出现的空洞补上，解决了重建点云时在patch的边界上出现空洞的问题。
以下，分别对不同目标操作下，本申请实施例提供的技术方案进行说明。
一、目标操作是置0操作。
可选的，目标位置是该待处理边界像素块中的，且与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置；或者，目标位置是该待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置。其中，目标有效像素所在的直线与待处理边界像素块的类型相关，具体示例可以参考下文。
目标有效像素，是指距离与有效像素边界的距离最远的有效像素，有效像素边界为有效像素与无效像素的界限。
例如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是正上方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是正下方，该情况下，待处理边界像素块中的目标有效像素是该待处理边界像素块中的最下方一行的像素。如图6所示，为可适用于该示例的一种目标位置的示意图。图6中是以待处理边界像素块是4*4的像素块，且预设阈值是2(具体是2个单位距离，其中一个单位距离是水平或竖直方向上相邻两个像素之间的距离)为例进行说明的。
再如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是左下方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是右上方， 该情况下，待处理边界像素块中的目标有效像素是该待处理边界像素块中的最右上方的一个或多个像素。如图7所示，为可适用于该示例的一种目标位置的示意图。其中，图7中的(a)图是以目标位置是该待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置为例进行说明，图7中的(b)图是以目标位置是该待处理边界像素块中的，且与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置为例进行说明的。并且，图7中，待处理边界像素块是4*4的像素块，且预设阈值是2(具体是2个单位距离，其中一个单位距离是是45度斜线方向上相邻两个像素之间的距离)。
又如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是正上方和左下方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是正下方和右上方，该情况下，待处理边界像素块中的目标有效像素是该待处理边界像素块中的最下方一行的像素，以及最右上方的一个或多个像素，如图8中的(a)图的阴影部分所示。预设像素位置如图8中的(b)图中的黑色部分所示。
其他示例与此类似，此处不再一一列举。
以下，基于所依据的空域相邻像素块不同，说明待处理边界像素块的类型(或待处理边界像素块中的无效像素在待处理边界像素块中的方位信息)的具体实现方式。
需要说明的是，这里描述的所依据的空域相邻像素块是指，确定待处理边界像素块的类型时，所依据的空域相邻像素块。而不应理解为待处理边界像素块所具有的空域相邻像素块。例如，可能存在一个待处理边界像素块的空域相邻像素块包括8个像素块，但是基于下述情况一时，仅依据该待处理边界像素块的正上方、正下方、正左方和正右方的像素块，来确定待处理边界像素块的类型。其他示例与此类似，此处不再一一列举。
情况一：待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻且位于待处理边界像素块的正上方、正下方、正左方和正右方的像素块。该情况下，待处理边界像素块中的无效像素在待处理边界像素块中的方位信息可以包括以下任一种：
方式1A：若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；该预设方向包括正上方、正下方、正左方和正右方中的其中一种或至少两种的组合。
具体的，若该预设方向是正上方，则可以将方式1A所描述的方位信息对应的待处理边界像素块类型称为类型1。若该预设方向是正下方，则可以将方式1A所描述的方位信息对应的待处理边界像素块类型称为类型2。若该预设方向是正左方，则可以将方式1A所描述的方位信息对应的待处理边界像素块类型称为类型7。若该预设方向是正右方，则可以将方式1A所描述的方位信息对应的待处理边界像素块类型称为类型8。
方式1B：若待处理边界像素块的正上方和正右方的像素块为无效像素块，且待处理边界像素块的正下方和正左方的像素块是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待 处理边界像素块中的右上方。示例的，该方位信息对应的待处理边界像素块类型称为类型3。
或者，若待处理边界像素块的正下方和正左方的像素块为无效像素块，且待处理边界像素块的正上方和正右方的像素块是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左下方。示例的，该方位信息对应的待处理边界像素块类型称为类型4。
或者，若待处理边界像素块的正上方和正左方的像素块为无效像素块，且待处理边界像素块的正下方和正右方的像素块是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左上方。示例的，该方位信息对应的待处理边界像素块类型称为类型5。
或者，若待处理边界像素块的正下方和正右方的像素块为无效像素块，且待处理边界像素块的正上方和正左方的像素块是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的右下方。示例的，该方位信息对应的待处理边界像素块类型称为类型6。
情况二：待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻的且位于待处理边界像素块的正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方的像素块。该情况下，若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方或右下方。
具体的：若该预设方向是右上方，则可以将该方位信息对应的待处理边界像素块类型称为类型9。若该预设方向是左下方，则可以将该方位信息对应的待处理边界像素块类型称为类型10。若该预设方向是左上方，则可以将该方位信息对应的待处理边界像素块类型称为类型11。若该预设方向是右下方，则可将该方位信息对应的待处理边界像素块类型称为类型12。
上述像素块的类型的索引(如上述类型1～12)、判别方式图、示意图以及描述信息等可以参考图9。其中，图9中的每个小方格表示一个像素块，最中心的标记有五角星的像素块表示待处理边界像素块，黑色标记的像素块表示无效像素块，白色标记的像素块表示有效像素块，斜线阴影标记的像素块表示有效像素块或无效像素块。
例如，图9所示的表格中的第一行中的判别方式图表示：当待处理边界像素块的空域相邻像素块中的正上方的像素块是无效像素块，且正下方、正左方和正右方的像素块均是有效像素块时，判定该待处理边界像素块的类型是类型1。该行中的示意图表示：待处理边界像素块的空域相邻像素块具有以下特征：正上方的像素块是无效像素块，且正下方、正左方和正右方的像素块均是有效像素块；且左上方、右上方、左下方和右下方的像素块是有效像素块或无效像素块。其他示例与此类似，此处不再一一列举。
情况三：待处理边界像素块的空域相邻像素块包括与待处理边界像素块相邻的且位于待处理边界像素块的左上方、右上方、左下方和右下方的像素块。该情况下，若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方和右下方其中一种或至少两种。
以下，基于待处理边界像素块的类型说明目标位置的具体实现方式。在此之前，首先说明以下几点：
第一，下文中的p[i]表示待译码点云经填充的占用图中的第i个边界像素块，p[i].type＝＝j表示边界像素块p[i]的类型的索引是j。
第二，为了便于描述，附图(如图10～图13)中对像素进行了编号，其中，这些附图中的每个小方格表示一个像素。另外下文中的具体示例分别以B0＝2、4或8为例进行说明。
第三，无论待处理边界像素块是哪一种类型，以及无论该类型对应一种还是多种处理方式，编码器和解码器均采用同一种方式对待处理边界块进行处理。
基于待处理边界像素块的类型说明目标位置的具体实现方式，可以包括：
如果p[i].type＝＝1，则令p(x,y)为B0*B0块中的一点，b l为去除强度参数，且b l∈[0,B0)；当p(x,y)满足x∈(0,B0],y∈(0,b l]时，p(x,y)＝0即将p点作为目标位置。
如果p[i].type＝＝2，则令p(x,y)为B0*B0块中的一点，b l为去除强度参数，且b l∈[0,B0)；当p(x,y)满足x∈(0,B0],y∈(B0-b l,B0]时，p(x,y)＝0即将p点作为目标位置。
如图10所示，为本申请实施例提供的一种确定目标位置的像素的示意图。
基于图10，如果p[i].type＝＝1，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{1}的像素。
当B0＝4时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}或者{1，2，3}的像素。
当B0＝8时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}、{1，2，3，4}、{1，2，3，4，5}、{1，2，3，4，5，6}或{1，2，3，4，5，6，7}。
基于图10，如果p[i].type＝＝2，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{2}的像素。
当B0＝4时，目标位置的像素可以是待处理边界像素块中的编号为{4}、{3，4}或者{2，3，4}的像素。
当B0＝8时，目标位置的像素可以是待处理边界像素块中的编号为{7}、{6，7}、{5，6，7}、{4，5，6，7}、{3，4，5，6，7}、{2，3，4，5，6，7}或{1，2，3，4，5，6，7}。
如果p[i].type＝＝3或者p[i].type＝＝9，则令p(x,y)为B0*B0块中的一点，x,y∈[0,B0)，b c为去除强度参数，且b c∈[-B0+2,B0-1]；当p(x,y)满足x-ky-b c+1<0时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如果p[i].type＝＝4或者p[i].type＝＝10，则令p(x,y)为B0*B0块中的一点，x,y∈[0,B0)，b c为去除强度参数，且b c∈[-B0+2,B0-1]；当p(x,y)满足x-ky+b c-1<0时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如图11所示，为本申请实施例提供的一种确定目标位置的像素的示意图。
基于图11，如果p[i].type＝＝3或9，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}或{1，2，3}的像素。
当B0＝4时，如果待处理边界像素块如B0＝4对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……7}的像素；如果待处理边界像素块如B0＝4对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……6}的像素。
当B0＝8时，如果待处理边界像素块如B0＝8对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……15}的像素；如果待处理边界像素块如B0＝8对应的第1个图，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……12}的像素。
基于图11，如果p[i].type＝＝4或10，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{3}、{2，3}或{1，2，3}的像素。
当B0＝4时，如果待处理边界像素块如B0＝4对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{7}、{6，7}、{5，6，7}……或{1，2，3……7}的像素；如果待处理边界像素块如B0＝4对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{6}、{5，6}、{4，5，6}……或{1，2，3……6}的像素。
当B0＝8时，如果待处理边界像素块如B0＝8对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{15}、{14，15}、{13，14，15}、{12，13，14，15}……或{1，2，3……15}的像素；如果待处理边界像素块如B0＝8对应的第1个图，目标位置的像素可以是待处理边界像素块中的编号为{11}、{11，12}、{10，11，12}……或{1，2，3……12}的像素。
如果p[i].type＝＝5或者p[i].type＝＝11，则令p(x,y)为B0*B0块中的一点，x,y∈[0,B0)，bc为去除强度参数，且b c∈[-B0+2,B0-1]。当p(x,y)满足x+ky-B0+b c<0时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如果p[i].type＝＝6，或者p[i].type＝＝12，则令p(x,y)为B0*B0块中的一点，x,y∈[0,B0)，bc为去除强度参数，且b c∈[-B0+2,B0-1]。当p(x,y)满足x+ky-B0-b c+2>0时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如图12所示，为本申请实施例提供的一种确定目标位置的像素的示意图。
基于图12，如果p[i].type＝＝5或11，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}或{1，2，3}的像素。
当B0＝4时，如果待处理边界像素块如B0＝4对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}……或{1，2……7}的像素；如果待处理边界像素块如B0＝4对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……6}的像素。
当B0＝8时，如果待处理边界像素块如B0＝8对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……15}的像素；如果待处理边界像素块如B0＝8对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}、{1，2，3}……或{1，2，3……12}的像素。
基于图12，p[i].type＝＝6或12，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{3}、{2，3}或{1，2，3}的像素。
当B0＝4时，如果待处理边界像素块如B0＝4对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{7}、{6，7}、{5，6，7}……或{1，2，3……7}的像素；如果待处理边界像素块如B0＝4对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{6}、{5，6}、{4，5，6}……或{1，2，3……6}的像素。
当B0＝8时，如果待处理边界像素块如B0＝8对应的第1个图，则目标位置的像素可以是待处理边界像素块中的编号为{15}、{14，15}、{13，14，15}、{12，13，14，15}……或{1，2，3……15}的像素；如果待处理边界像素块如B0＝8对应的第2个图或第3个图，则目标位置的像素可以是待处理边界像素块中的编号为{12}、{11，12}、{10，11，12}……或{1，2，3……12}的像素。
如果p[i].type＝＝7，则令p(x,y)为B0*B0块中的一点，b l为去除强度参数，且b l∈[0,B0)。当p(x,y)满足x∈(B0-b l,B0],y∈(0,B0]时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如果p[i].type＝＝8，则令p(x,y)为B0*B0块中的一点，b l为去除强度参数，且b l∈[0,B0)。当p(x,y)满足x∈(0,b l],y∈(0,B0]时，p(x,y)＝0即将p点作为目标位置。其中，k>0。
如图13所示，为本申请实施例提供的一种确定目标位置的像素的示意图。
基于图13，如果p[i].type＝＝7，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{2}或{1，2}的像素。
当B0＝4时，目标位置的像素可以是待处理边界像素块中的编号为{4}、{3，4}……或{1，2……4}的像素。
当B0＝8时，目标位置的像素可以是待处理边界像素块中的编号为{8}、{7，8}……或{1，2……8}的像素。
基于图13，p[i].type＝＝8，那么：
当B0＝2时，目标位置的像素可以是待处理边界像素块中的编号为{1}或{1，2}的像素。
当B0＝4时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}或{1，2……4}的像素。
当B0＝8时，目标位置的像素可以是待处理边界像素块中的编号为{1}、{1，2}……或{1，2……8}的像素。
上文中描述的目标位置的像素的具体实现方式仅为示例，实际实现时不限于此。
二、目标操作是置1操作。
可选的，目标位置是该待处理边界像素块中的，且与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置；或者，目标位置是该待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置。其中，目标有效像素所在的直线与待处理边界像素块的类型相关，具体示例可以参考下文。
目标有效像素，是指距离与有效像素边界的距离最远的有效像素，有效像素边界为有效像素与无效像素的界限。
例如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是正上方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是正下方，该情况下，待处理边界像素块中的目标有效像素是该待处理边界像素块中的最下方一行的像素。如图14所示，为可适用于该示例的一种目标位置的示意图。图14中是以待处理边界像素块是4*4的像素块，且预设阈值是2(具体是2个单位距离，其中一个单位距离是水平或竖直方向上相邻两个像素之间的距离)为例进行说明的。
如图14所示，待处理边界像素块中的目标有效像素为图14中的a图的第一行像素和第二行像素，采用对应的目标处理方式将待处理边界像素块中的目标位置的像素的值置1，得到经置1的边界像素块包括：将图14中的a图第二行像素的值置1，经置1的边界像素块如图14中的b图所示；或者将图14中的a图的第一行和第二行像素的值均置为1，经置1的边界像素块如图14中的c图所示。
再如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是左下方，则待处理边界像素块中的有效像素在待处理边界像素块中的方位信息是右上方，该情况下，待处理边界像素块中的目标有效像素是该待处理边界像素块中的最右上方的一个或多个像素。如图15所示，为可适用于该示例的一种目标位置的示意图。其中，图15中的(a)图是以目标位置是该待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置为例进行说明，图15中的(e)图是以目标位置是该待处理边界像素块中的，且与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置为例进行说明的。并且，图15中，待处理边界像素块是尺寸为4*4的像素块，且预设阈值是2(具体是2个单位距离，其中一个单位距离是是45度斜线方向上相邻两个像素之间的距离)。
图15中的(a)图是以目标位置是该待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于2的无效像素所在的位置为例进行说明，将与目标有效像素所在的直线之间的距离小于或等于2的部分无效像素或者全部无效像素的像素值置1，以得到经置1的边界像素块，经置1后的边界像素块如图15中的(b)、(c)和(d)图所示；图15中的(e)图是以目标位置是该待处理边界像素块中的， 且与目标有效像素之间的距离小于或等于2的无效像素所在的位置为例进行说明的。将与目标有效像素之间的距离小于或等于2的部分无效像素或者全部无效像素的像素值置1，以得到置1后的边界像素块，经置1的边界像素块如图15中的(f)图和(g)图所示。
又如，若待处理边界像素块中的无效像素在待处理边界像素块中的方位信息是正上方和左下方，待处理边界像素块中的目标无效像素是该待处理边界像素块中的下方倒数第二行的像素，以及右上方的一个或多个像素，如图16中的(a)图的阴影部分所示。目标位置如图16中的(b)图中的白色部分所示。
其他示例与此类似，此处不再一一列举。
需要说明的是，当目标操作是置1操作时，待处理边界像素块的类型的具体实现方式，可以参考当目标操作是置0时的相应的具体实现方式，此处不再赘述。
另外需要说明的是，在一些实施例中，当目标操作是置0操作时，目标位置包括：待处理边界像素块中的，从距离目标有效像素最远距离Lmax的位置至距离目标有效像素预设距离L1的位置。Lmax＞L1。相应的，当目标操作是置1操作时，目标位置包括：待处理边界像素块中的，从距离目标有效像素最近距离Lmin(如该距离可以是0或1等)的位置至距离目标有效像素预设距离L2的位置。Lmin＜L1。
例如，参见图9和图10，假设待处理边界像素块是图9中的类型1，且B0＝4；那么：以目标位置是待处理边界像素块中的1行像素为例，当目标操作是置0时，目标位置是图10中的编号为1的像素所在的位置；当目标操作是置1时，目标位置是图10中的编号为4的像素所在的位置。以目标位置是待处理边界像素块中的2行像素为例，当目标操作是置0时，目标位置是图10中的编号为1、2的像素所在的位置；当目标操作是置1时，目标位置是图10中的编号3、4的像素所在的位置。其他示例不再一一列举。
三、目标操作是膨胀操作。
以下，以基本单位是一个像素为例对膨胀操作进行说明。
膨胀操作的卷积核可以是任意形状和大小，如正方形或者圆形，具体可以参考现有技术。卷积核一般会定义一个锚点，该锚点一般为卷积核的中心点。作为一个示例，卷积核可以为图17中的任意一种。图17中，白色方块表示像素值为0的像素，阴影方块表示像素值为1的像素，五角星所在的像素块为锚点。图17中的卷积核的尺寸为5*5(其中R＝5)。
具体实现的过程中，可以取占用图中的像素p[x][y]，将图17中某一个卷积核(具体是哪一种可以编码器和解码器预定义的，当然本申请实施例不限于此)的锚点与p[x][y]对齐，若卷积核中阴影方块所示位置在p[x][y]像素点对应邻域点中有至少一个像素点的像素值为1，则q[x][y]取值为1，否则q[x][y]取值为0。
可以理解的，卷积核的半径决定膨胀操作影响像素的多少。卷积核的半径越大，经膨胀的像素点越多；卷积核的半径越小，经膨胀的像素点越少。
以下，说明上述S102A的具体实现方式。下文中的具体示例，通常是以目标操作是置0操作为例进行说明的。
可选的，上述S102A可以包括：根据边界像素块的多种类型与多种处理方式之间 的映射关系，确定待处理边界像素块的类型对应的处理方式；若待处理边界像素块的类型对应一种处理方式，则将待处理边界像素块的类型对应的处理方式作为目标处理方式；或者，若待处理边界像素块的类型对应多种处理方式，则将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式；采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
在该可选的实现方式中，编码器和解码器可以预定义(如通过协议预定义)边界像素块的多种类型与多种处理方式之间的映射关系，例如预定义边界像素块的多种类型的标识信息与多种处理方式的标识信息之间的映射关系。
本申请实施例中对上述映射关系的具体体现形式不进行限定，例如可以是表格，或者是公式，或者是根据条件进行逻辑判断(例如if else或者switch操作等)等。下文中主要以映射关系的具体体现表格为例进行说明。基于此，执行S102A时，译码器可以通过查表，得到待处理边界像素块的类型对应的处理方式。可以理解的，上述映射关系具体体现在一个或多个表格中，本申请实施例对此不进行限定。为了便于描述，本申请实施例均以这些表格具体体现在一个表格中为例进行说明。在此统一说明，下文不再赘述。基于此，上述S102A具体可以包括：根据待处理边界像素块的类型查表，得到待处理边界像素块的类型对应的处理方式，该表包括边界像素块的多种类型与多种处理方式之间的映射关系。
如果待处理边界像素块对应一种处理方式，则编码器和解码器均可以通过预定义的上述映射关系，获得目标处理方式。因此，该情况下，编码器可以不用向解码器发送用于表示目标处理方式的标识信息，这样可以节省码流传输开销。例如，根据上文中的描述，基于图10，假设待处理边界像素块的类型的索引是1，且B0＝4，则该类型对应的一种处理方式(即目标处理方式)可以是：将待处理边界像素块中的编号为{1}的像素置0。
如果待处理边界像素块对应多种处理方式，则编码器可以从该多种处理方式中选择一种处理方式作为目标处理方式。例如，根据待处理边界像素块在填充之前的像素块中的像素值为0的像素的位置，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式。例如，根据上文中的描述，基于图10，假设待处理边界像素块的类型的索引是1，则该类型对应的多种处理方式可以是：将待处理边界像素块中的编号为{1}的像素置0，以及将待处理边界像素块中的编号为{1，2}的像素置0。目标处理方式可以是将待处理边界像素块中的编号为{1}的像素置0，或者将待处理边界像素块中的编号为{1，2}的像素置0。
可选的，将待处理边界像素块的类型对应多种的处理方式的其中一种处理方式作为目标处理方式，可以包括：根据待处理边界像素块在填充之前的像素块中的像素值为0的像素的位置，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式。其中，所选择的目标处理方式使得待处理边界像素块中的最多的无效像素置0。
例如，如图18所示，为本申请实施例提供的两种类型为1的待处理边界像素块(即无效像素在待处理边界像素块内部的正上方)在填充之前的像素块的示意图。其中，如果待处理边界像素块在填充之前如图18中的(a)图所示，即第1行中的像素为无 效像素，则目标处理方式可以是将待处理边界像素块中的编号为{1}的像素置0。如果待处理边界像素块在填充之前如图18中的(b)图所示，即第1行和第2行中的像素为无效像素，则目标处理方式可以是将待处理边界像素块中的编号为{1，2}的像素置0。其中，图18中是以待处理边界像素块的大小是4*4为例进行说明的。其他示例的原理与此类似，此处不再赘述。
可选的，如果待处理边界像素块对应多种处理方式，则编码器可以将标识信息编入码流，该标识信息表示待处理边界像素块的目标处理方式。该情况下，对于解码器来说，上述S102A可以包括：根据待处理边界像素块的类型，解析码流，以得到该标识信息；然后采用目标处理方式对待处理边界像素块进行目标操作。
可以理解的，如果待处理边界像素块的空域相邻像素块包括8个，则该待处理边界像素块的空域相邻像素块可能的组合共有2 8种，这2 8种的其中一种或者至少两种可以作为一种类型，例如如图9所示的若干种类型。另外，除了上文中所列举的边界像素块的类型之外，边界像素块还可以被归为其他类型。实际实现的过程中，由于待处理边界像素块的空域相邻像素块可能的组合比较多，因此，可以选择出现概率比较高的类型，或者执行本申请实施例提供的置0处理后对编码效率增益贡献较大的类型，来执行本申请实施例提供的技术方案，对于其他类型，可以不执行本申请实施例提供的技术方案。基于此，对于解码器来说，可以根据待处理边界像素块的类型(具体是指按照本申请实施例提供的技术方案进行编解码的边界像素块的类型，或者对应多种处理方式的边界像素块的类型)，确定是否解析码流。其中，这里的码流是指携带目标处理方式的标识信息的码流。
例如，假设编码器和解码器预定义：针对如图9所示的各种类型的边界像素块，按照本申请实施例提供的技术方案进行编解码；那么，对于解码器来说，当确定一个待处理边界像素块的类型是图9中所示的其中一种类型时，解析码流，以得到该类型对应的目标处理方式；当该待处理边界像素块的类型不是图9中所示的类型时，不解析码流。这样，不需要在码流中传输每个待处理边界像素块的每种类型以及每种类型对应的目标处理方式，因此可以节省码流传输开销。
可选的，基于上述方式A，如果待处理边界像素块的类型对应的多种处理方式，则译码器可以根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式；目标空域相邻像素块是待处理边界像素块的为无效像素块的空域相邻像素块。
也就是说，本申请实施例支持基于预设阈值以及待处理边界像素块的空域相邻像素块中的非空无效像素块决定的距离值，确定目标位置。
进一步可选的，根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式，包括：如果目标空域相邻像素块是空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第一处理方式作为目标处理方式；如果目标空域相邻像素块是非空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第二处理方式作为目标处理方式。其中，当目标操作是置0操作时，第一处理方式对应的目标位置的范围大于第二处理方式对应的目标位置的范围。当目标操作是置1操 作时，第一处理方式对应的目标位置的范围小于第二处理方式对应的目标位置的范围。当目标操作是膨胀操作时，第一处理方式对应的卷积核的半径小于第二处理方式对应的卷积核的半径。
可以理解的是，如果待处理边界像素块的空域相邻像素块中包括非空无效像素块，说明两个不同patch占用图(即待处理边界像素块所在的patch占用图和该非空无效像素块所在的patch占用图)在待处理边界像素块附近相对更加密实地被打包在一起。由此，对应的深度图像和纹理图像在此处的局部不连续性，通常比附近不存在非空无效相邻像素块的边界像素块处要大。从而，此处的编码误差也会相对较大。因此，为了避免在重构的点云中出现空洞，此处的置0操作要相对更加保守(即在相应方向上将相对较少数目的像素置为0)，或者置1操作相对更加激进(即置1操作对应的目标位置的范围相对更大，或者膨胀操作对应的卷积核的半径相对更大)。例如，设B0＝4，当执行置零操作的时候，如果待处理像素块的正上方相邻像素块为非空无效像素块的时候，置待处理像素块的最顶部一行的像素为零；如果待处理像素块的正上方相邻像素块为空无效像素块的时候，置待处理像素块的最顶部两行的像素为零。当执行置1操作的时候，如果待处理像素块的正上方相邻像素块为非空无效像素块，则与有效像素距离为2的无效像素均置为1；如果待处理像素块的正上方相邻像素块为空无效像素块，则与有效像素距离为1的无效像素均置为1。
需要说明的是，基于上述方式B，当目标操作是置0操作时，第三处理方式对应的目标位置的范围小于第四处理方式对应的目标位置的范围。当目标操作是置1操作时，第三处理方式对应的目标位置的范围大于第四处理方式对应的目标位置的范围。当目标操作是膨胀操作时，第三处理方式对应的卷积核的半径大于第四处理方式对应的卷积核的半径。其中，第三处理方式是第一待处理像素块的类型对应的目标处理方式，第四处理方式是第二待处理像素块的类型对应的目标处理方式。第一待处理像素块的预设方位的空域相邻像素块是空无效像素块，且第二待处理像素块的该预设方位的空域相邻像素块是非空无效像素块。
如图19所示，为本申请实施例提供的一种码流结构的示意图。图19中的每个带箭头的连线表示一个边界像素块与该边界像素块的目标处理方式的标识信息之间的对应关系。图19中的数字表示边界像素块的索引。
上文中描述了基于预定义的边界像素块的类型与处理方式之间的映射关系，确定待处理边界像素块的目标处理方式的技术方案。可替换的，编码器可以动态确定待处理边界像素块的类型对应的目标处理方式，然后将目标处理方式的相关信息编入码流，该情况下，解码器可以通过解析该码流，获得该目标处理方式。作为一个示例，目标处理方式的相关信息可以包括：被置0或置1的像素的索引(如坐标值等)。
如图20所示，为本申请实施例提供的一种点云编码方法的流程示意图。本实施例的执行主体可以是编码器。该方法可以包括：
S301：确定指示信息，该指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；目标编码方法包括本申请实施例提供的任一种点云编码方法，例如可以是图5所示的点云译码方法，且这里的译码具体是指编码。
具体实现的过程中，编码方法可以有至少两种，该至少两种的其中一种可以是本 申请实施例提供的任一种点云编码方法，其他种可以是现有技术或未来提供的点云编码方法。
可选的，该指示信息具体可以是目标点云编码/解码方法的索引。具体实现的过程中，编码器和解码器可以预先约定编码器/解码器所支持的至少两种点云编码/解码方法的索引，然后，在编码器确定目标编码方法之后，将目标编码方法的索引或该目标编码方法对应的解码方法的索引作为指示信息编入码流。本申请实施例对编码器如何确定目标编码方法是编码器所支持的至少两种编码方法中的哪一种不进行限定。
S302：将该指示信息编入码流。可选的，该指示信息是帧级别的信息。
本实施例提供了一种选择目标编码方法的技术方案，该技术方案可以应用于编码器支持至少两种点云编码方法的场景中。
如图21所示，为本申请实施例提供的一种点云解码方法的流程示意图。本实施例的执行主体可以是解码器。该方法可以包括：
S401：解析码流，以得到指示信息，该指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；目标解码方法包括本申请实施例提供的任一种点云解码方法，例如可以是图5所示的点云译码方法，且这里的译码具体是指解码。具体是与图20中所描述的编码方法相对应的解码方法。其中，该指示信息是帧级别的信息。
S402：当该指示信息用于指示按照目标解码方法对待解码点云的占用图进行处理时，按照目标解码方法对待解码点云的占用图进行处理。其中，具体的处理过程可以参考上文。
本实施例提供的点云解码方法与图20提供的点云编码方法相对应。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对编码器/解码器进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图22所示，为本申请实施例提供的一种译码器230的示意性框图。译码器230具体可以是编码器或解码器。译码器230可以包括分类模块2301和重构模块2302。
当译码器230是编码器时，在一个示例中，分类模块2301可以对应图2中的占用图滤波模块112中的分类子模块，相应地，重构模块2302可以对应于图2中的一个或多个模块的组合，例如，重构模块2302可以对应于图2中的占用图滤波模块112中的预处理子模块，以及点云重构模块111。
当译码器230是解码器时，在一个示例中，分类模块2301可以对应图2中的占用 图滤波模块208中的分类子模块，相应地，重构模块2302可以对应于图4中的一个或多个模块的组合，例如，重构模块2302可以对应于图4中的占用图滤波模块208中的预处理子模块，以及点云重构模块111。
可选的，分类模块2301，用于基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型；其中，无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；目标patch占用图是待处理边界像素块所属的patch占用图。重构模块2302，用于根据待处理边界像素块的类型，重构待译码点云。
可选的，分类模块2301还用于当待译码点云的占用图的当前边界像素块的空域相邻像素块中的无效像素块的个数大于或等于预设阈值时，确定当前边界像素块为待处理边界像素块。
可选的，分类模块2301在基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型的方面，具体用于：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息；其中，不同类型的边界像素块对应不同的方位信息。
可选的，如果待处理边界像素块的预设方位的空域相邻像素块是无效像素块，则确定待处理边界像素块中的无效像素在待处理边界像素块中的预设方位；预设方位是正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方中的其中一种或者至少两种的组合。
可选的，重构模块2302具体用于：根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块，例如该步骤可以认为是由图2中的占用图滤波模块112或图4中的占用图滤波模块208中的预处理子模块执行的；根据经处理过的占用图，重构待译码点云，经处理过的占用图包括经目标操作的像素块，例如该步骤可以认为是由图2中的点云重构模块111或图4中的点云重构模块205执行的。
其中，重构模块2302在执行根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块的方面，具体用于：当目标操作是置0操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置0，得到经置0的像素块，不同处理方式对应的目标位置不同；或者，当目标操作是置1操作时，根据待处理边界像素块的类型，采用对应的目标处理方式将待处理边界像素块的目标位置的像素的值置1，得到经置1的像素块，不同处理方式对应的目标位置不同；或者，当目标操作是膨胀操作时，根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行膨胀操作，得到经膨胀操作后的像素块，不同处理方式的卷积核的半径不同。
可选的，重构模块2302在执行根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块的方面，具体用于：根据边界像素块的多种类型与多种处理方式之间的映射关系，确定待处理边界像 素块的类型对应的处理方式；若待处理边界像素块的类型对应一种处理方式，则将待处理边界像素块的类型对应的处理方式作为目标处理方式；或者，若待处理边界像素块的类型对应多种处理方式，则将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式；采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
可选的，重构模块2302在根据待处理边界像素块的类型，采用对应的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块的方面，具体用于：根据待处理边界像素块的类型查表，得到待处理边界像素块的类型对应的处理方式，表包括边界像素块的多种类型与多种处理方式之间的映射关系；若待处理边界像素块的类型对应一种处理方式，则将待处理边界像素块的类型对应的处理方式作为目标处理方式；或者，若待处理边界像素块的类型对应多种处理方式，则将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式；采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
可选的，重构模块2302在将待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为目标处理方式的方面，具体用于：根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式；目标空域相邻像素块是待处理边界像素块的为无效像素块的空域相邻像素块。
可选的，重构模块2302在根据待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为目标处理方式的方面，具体用于：如果目标空域相邻像素块是空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第一处理方式作为目标处理方式；如果目标空域相邻像素块是非空无效像素块，则从待处理边界像素块的类型对应的多种处理方式中选择第二处理方式作为目标处理方式；其中，当目标操作是置0操作时，第一处理方式对应的目标位置的范围大于第二处理方式对应的目标位置的范围；或者，当目标操作是置1操作时，第一处理方式对应的目标位置的范围小于第二处理方式对应的目标位置的范围；或者，当目标操作是膨胀操作时，第一处理方式对应的卷积核的半径小于第二处理方式对应的卷积核的半径。
可选的，待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻且位于待处理边界像素块的正上方、正下方、正左方和正右方的像素块；若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括正上方、正下方、正左方和正右方中的其中一种或至少两种的组合；或者，若待处理边界像素块的正上方和正右方的像素块为无效像素块，且待处理边界像素块的正下方和正左方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的右上方；或者，若待处理边界像素块的正下方和正左方的像素块为无效像素块，且待处理边界像素块的正上方和正右方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左下方；或者，若待处理边界像素块的正上方和正左方的像素块为无 效像素块，且待处理边界像素块的正下方和正右方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的左上方；或者，若待处理边界像素块的正下方和正右方的像素块为无效像素块，且待处理边界像素块的正上方和正左方的像素块是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的右下方。
可选的，待处理边界像素块的空域相邻像素块包括与待处理边界像素块相邻的且位于待处理边界像素块的左上方、右上方、左下方和右下方的像素块；若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方和右下方其中一种或至少两种。
可选的，待处理边界像素块的空域相邻像素块包括：与待处理边界像素块相邻的且位于待处理边界像素块的正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方的像素块；若待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则方位信息是：待处理边界像素块中的无效像素位于待处理边界像素块中的预设方向；预设方向包括左上方、右上方、左下方或右下方。
可选的，当目标操作是置0操作时，目标位置是待处理边界像素块中的，与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置；或者，目标位置是待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置；直线与待处理边界像素块的类型相关；或者，当目标操作是置1操作时，目标位置是待处理边界像素块中的，与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置；或者，目标位置是待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置；直线与待处理边界像素块的类型相关。
可选的，分类模块2301在基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块的类型的方面，具体用于：基于待处理边界像素块的空域相邻像素块是否是无效像素块，确定待处理边界像素块中的无效像素在待处理边界像素块中的方位信息；基于待处理边界像素块的为无效像素块的空域相邻像素块是否是空无效像素块，以及确定的方位信息，确定待处理边界像素块的类型。
可选的，待译码点云的占用图包括第一待处理边界像素块和第二待处理边界像素块；第一待处理边界像素块的类型是第一类型，第二待处理边界像素块的类型是第二类型；第一类型与第二类型所对应的方位信息相同；且第一待处理边界像素块的空域相邻像素块中预设方位的无效像素块是空无效像素块，第二待处理边界图像的空域相邻像素块中预设方位的无效像素块是非空无效像素块。
可选的，译码器230是编码器，若待处理边界像素块的类型对应多种处理方式；如图23所示，该编码器还包括：辅助信息编码模块2303，用于将标识信息编入码流，该标识信息表示待处理边界像素块的目标处理方式。
可选的，译码器230是解码器，若待处理边界像素块的类型对应多种处理方式， 重构模块2302在采用目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块的方面，具体用于：根据待处理边界像素块的类型，解析码流，以得到标识信息；标识信息表示目标处理方式；采用标识信息所指示的目标处理方式对待处理边界像素块进行目标操作，得到经目标操作的像素块。
可选的，译码器230是解码器，如图24所示，解码器还包括：辅助信息解码模块2304，用于解析码流，以得到待译码点云的待处理边界像素块的尺寸信息；分类模块2301具体用于根据尺寸信息对待解码点云的占用图进行划分，得到一个或多个待处理边界像素块。
如图25所示，为本申请实施例提供的一种编码器250的示意性框图。编码器250可以包括辅助信息编码模块2501。例如，编码器250可以是图2中的编码器100，该情况下，辅助信息编码模块2501可以是辅助信息编码模块108。其中，辅助信息编码模块2501，用于确定指示信息，以及，将该指示信息编入码流。该指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；目标编码方法包括上文提供的任意一种点云译码方法(具体是点云编码方法)，如图5所示的点云译码方法。
可以理解的，具体实现的过程中，编码器250还包括占用图滤波模块2502和点云重构模块2503，用于按照目标编码方法对待编码点云的占用图进行处理。其中，占用图滤波模块2502所执行的步骤可以参考上述分类模块2301和预处理子模块所执行的步骤，点云重构模块2503所执行的步骤可以参考上述点云重构模块所执行的步骤，此处不再赘述。
如图26所示，为本申请实施例提供的一种解码器260的示意性框图。解码器260可以包括：辅助信息解码模块2601、占用图滤波模块2602和点云重构模块2603。其中，辅助信息解码模块2601，用于解析码流，以得到指示信息，指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；目标解码方法包括上文提供的任意一种点云译码方法(具体是点云解码方法)，如图5所示的点云译码方法。占用图滤波模块2602和点云重构模块2603，用于当该指示信息用于指示按照目标解码方法对待解码点云的占用图进行处理时，按照目标解码方法对待解码点云的占用图进行处理，具体处理过程可以参考上文，此处不再赘述。占用图滤波模块2602所执行的步骤可以参考上述分类模块2301和预处理子模块所执行的步骤，点云重构模块2603所执行的步骤可以参考上述点云重构模块所执行的步骤，此处不再赘述。
可以理解的，本申请实施例提供的译码器230或编码器250或解码器260中的各模块为实现上文提供的相应的方法中所包含的各种执行步骤的功能主体，即具备实现完整实现本申请图像滤波方法中的各个步骤以及这些步骤的扩展及变形的功能主体，具体请参见上文中相应方法的介绍，为简洁起见，本文将不再赘述。
图27为用于本申请实施例的编码设备或解码设备(简称为译码设备270)的一种实现方式的示意性框图。其中，译码设备270可以包括处理器2710、存储器2730和总线系统2750。其中，处理器2710和存储器2730通过总线系统2750相连，该存储器2730用于存储指令，该处理器2710用于执行该存储器2730存储的指令，以执行本申请描述的各种点云译码方法。为避免重复，这里不再详细描述。
在本申请实施例中，该处理器2710可以是中央处理单元(central processing unit， CPU)，该处理器2710还可以是其他通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该存储器2730可以包括ROM设备或者RAM设备。任何其他适宜类型的存储设备也可以用作存储器2730。存储器2730可以包括由处理器2710使用总线2750访问的代码和数据2731。存储器2730可以进一步包括操作系统2733和应用程序2735，该应用程序2735包括允许处理器2710执行本申请描述的视频编码或解码方法(尤其是本申请描述的基于当前像素块的块尺寸对当前像素块进行滤波的方法)的至少一个程序。例如，应用程序2735可以包括应用1至N，其进一步包括执行在本申请描述的视频编码或解码方法的视频编码或解码应用(简称视频译码应用)。
该总线系统2750除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线系统2750。
可选的，译码设备270还可以包括一个或多个输出设备，诸如显示器2770。在一个示例中，显示器2770可以是触感显示器，其将显示器与可操作地感测触摸输入的触感单元合并。显示器2770可以经由总线2750连接到处理器2710。
本领域技术人员能够领会，结合本文公开描述的各种说明性逻辑框、模块和算法步骤所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件来实施，那么各种说明性逻辑框、模块、和步骤描述的功能可作为一或多个指令或代码在计算机可读媒体上存储或传输，且由基于硬件的处理单元执行。计算机可读媒体可包含计算机可读存储媒体，其对应于有形媒体，例如数据存储媒体，或包括任何促进将计算机程序从一处传送到另一处的媒体(例如，根据通信协议)的通信媒体。以此方式，计算机可读媒体大体上可对应于非暂时性的有形计算机可读存储媒体，或通信媒体，例如信号或载波。数据存储媒体可为可由一或多个计算机或一或多个处理器存取以检索用于实施本申请中描述的技术的指令、代码和/或数据结构的任何可用媒体。计算机程序产品可包含计算机可读媒体。
作为实例而非限制，此类计算机可读存储媒体可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来存储指令或数据结构的形式的所要程序代码并且可由计算机存取的任何其它媒体。并且，任何连接被恰当地称作计算机可读媒体。举例来说，如果使用同轴缆线、光纤缆线、双绞线、数字订户线(DSL)或例如红外线、无线电和微波等无线技术从网站、服务器或其它远程源传输指令，那么同轴缆线、光纤缆线、双绞线、DSL或例如红外线、无线电和微波等无线技术包含在媒体的定义中。但是，应理解，所述计算机可读存储媒体和数据存储媒体并不包括连接、载波、信号或其它暂时媒体，而是实际上针对于非暂时性有形存储媒体。如本文中所使用，磁盘和光盘包含压缩光盘(CD)、激光光盘、光学光盘、DVD和蓝光光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光以光学方式再现数据。以上各项的组合也应包含在计算机可读媒体的范围内。
可通过例如一或多个数字信号处理器(DSP)、通用微处理器、专用集成电路(ASIC)、现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指前述结构或适合于实施本文中所 描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文中所描述的各种说明性逻辑框、模块、和步骤所描述的功能可以提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或者并入在组合编解码器中。而且，所述技术可完全实施于一或多个电路或逻辑元件中。在一种示例下，编码器100及解码器200中的各种说明性逻辑框、单元、模块可以理解为对应的电路器件或逻辑元件。
本申请的技术可在各种各样的装置或设备中实施，包含无线手持机、集成电路(IC)或一组IC(例如，芯片组)。本申请中描述各种组件、模块或单元是为了强调用于执行所揭示的技术的装置的功能方面，但未必需要由不同硬件单元实现。实际上，如上文所描述，各种单元可结合合适的软件和/或固件组合在编码解码器硬件单元中，或者通过互操作硬件单元(包含如上文所描述的一或多个处理器)来提供。
以上所述，仅为本申请示例性的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。

Claims (46)

  1. 一种点云译码方法，其特征在于，包括：
    基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型；其中，所述无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，所述无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；所述目标patch占用图是所述待处理边界像素块所属的patch占用图；
    根据所述待处理边界像素块的类型，重构所述待译码点云。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    当所述待译码点云的占用图的当前边界像素块的空域相邻像素块中的无效像素块的个数大于或等于预设阈值时，确定所述当前边界像素块为所述待处理边界像素块。
  3. 根据权利要求1或2所述的方法，其特征在于，所述基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型包括：
    基于所述待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的方位信息；
    其中，不同类型的边界像素块对应不同的方位信息。
  4. 根据权利要求3所述的方法，其特征在于，如果所述待处理边界像素块的预设方位的空域相邻像素块是无效像素块，则确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的所述预设方位；其中，所述预设方位是正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方中的其中一种或者至少两种的组合。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，所述根据所述待处理边界像素块的类型，重构所述待译码点云，包括：
    根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块；并根据经处理过的占用图，重构所述待译码点云，所述经处理过的占用图包括所述经目标操作的像素块；
    其中，所述根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块，包括：
    当所述目标操作是置0操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式将所述待处理边界像素块的目标位置的像素的值置0，得到经所述置0的像素块，不同处理方式对应的目标位置不同；
    或者，当所述目标操作是置1操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式将所述待处理边界像素块的目标位置的像素的值置1，得到经所述置1的像素块，不同处理方式对应的目标位置不同；
    或者，当所述目标操作是膨胀操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行膨胀操作，得到经膨胀操作后的像素块，不同处理方式的卷积核的半径不同。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述待处理边界像素块的 类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块，包括：
    根据边界像素块的多种类型与多种处理方式之间的映射关系，确定所述待处理边界像素块的类型对应的处理方式；
    若所述待处理边界像素块的类型对应一种处理方式，则将所述待处理边界像素块的类型对应的处理方式作为所述目标处理方式；或者，若所述待处理边界像素块的类型对应多种处理方式，则将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式；
    采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  7. 根据权利要求5所述的方法，其特征在于，所述根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块，包括：
    根据待处理边界像素块的类型查表，得到所述待处理边界像素块的类型对应的处理方式，所述表包括边界像素块的多种类型与多种处理方式之间的映射关系；
    若所述待处理边界像素块的类型对应一种处理方式，则将所述待处理边界像素块的类型对应的处理方式作为所述目标处理方式；或者，若所述待处理边界像素块的类型对应多种处理方式，则将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式；
    采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  8. 根据权利要求6或7所述的方法，其特征在于，所述将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式，包括：
    根据所述待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从所述待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为所述目标处理方式；所述目标空域相邻像素块是所述待处理边界像素块的为无效像素块的空域相邻像素块。
  9. 根据权利要求8所述的方法，其特征在于，所述根据所述待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从所述待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为所述目标处理方式，包括：
    如果所述目标空域相邻像素块是空无效像素块，则从所述待处理边界像素块的类型对应的多种处理方式中选择第一处理方式作为所述目标处理方式；
    如果所述目标空域相邻像素块是非空无效像素块，则从所述待处理边界像素块的类型对应的多种处理方式中选择第二处理方式作为所述目标处理方式；
    其中，当所述目标操作是置0操作时，所述第一处理方式对应的目标位置的范围大于所述第二处理方式对应的目标位置的范围；
    或者，当所述目标操作是置1操作时，所述第一处理方式对应的目标位置的范围小于所述第二处理方式对应的目标位置的范围；
    或者，当所述目标操作是膨胀操作时，所述第一处理方式对应的卷积核的半径小 于所述第二处理方式对应的卷积核的半径。
  10. 根据权利要求3或4所述的方法，其特征在于，所述待处理边界像素块的空域相邻像素块包括：与所述待处理边界像素块相邻且位于所述待处理边界像素块的正上方、正下方、正左方和正右方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括正上方、正下方、正左方和正右方中的其中一种或至少两种的组合；
    或者，若所述待处理边界像素块的正上方和正右方的像素块为无效像素块，且所述待处理边界像素块的正下方和正左方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的右上方；
    或者，若所述待处理边界像素块的正下方和正左方的像素块为无效像素块，且所述待处理边界像素块的正上方和正右方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的左下方；
    或者，若所述待处理边界像素块的正上方和正左方的像素块为无效像素块，且所述待处理边界像素块的正下方和正右方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的左上方；
    或者，若所述待处理边界像素块的正下方和正右方的像素块为无效像素块，且所述待处理边界像素块的正上方和正左方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的右下方。
  11. 根据权利要求3或4所述的方法，其特征在于，所述待处理边界像素块的空域相邻像素块包括与所述待处理边界像素块相邻的且位于所述待处理边界像素块的左上方、右上方、左下方和右下方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括左上方、右上方、左下方和右下方其中一种或至少两种。
  12. 根据权利要求3或4所述的方法，其特征在于，所述待处理边界像素块的空域相邻像素块包括：与所述待处理边界像素块相邻的且位于所述待处理边界像素块的正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括左上方、右上方、左下方或右下方。
  13. 根据权利要求5至9任一项所述的方法，其特征在于，
    当所述目标操作是置0操作时，所述目标位置是所述待处理边界像素块中的，与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置；或者，所述目标位置是所述待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置；所述直线与所述待处理边界像素块的类型相 关；
    或者，当所述目标操作是置1操作时，所述目标位置是所述待处理边界像素块中的，与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置；或者，所述目标位置是所述待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置；所述直线与所述待处理边界像素块的类型相关。
  14. 根据权利要求1、2、4～13任一项所述的方法，其特征在于，所述基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型包括：
    基于所述待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的方位信息；
    基于所述待处理边界像素块的为无效像素块的空域相邻像素块是否是空无效像素块，以及所述确定的方位信息，确定所述待处理边界像素块的类型。
  15. 根据权利要求14所述的方法，其特征在于，所述待译码点云的占用图包括第一待处理边界像素块和第二待处理边界像素块；所述第一待处理边界像素块的类型是第一类型，所述第二待处理边界像素块的类型是第二类型；所述第一类型与所述第二类型所对应的方位信息相同；且所述第一待处理边界像素块的空域相邻像素块中预设方位的无效像素块是空无效像素块，所述第二待处理边界像素块的空域相邻像素块中所述预设方位的无效像素块是非空无效像素块。
  16. 根据权利要求6至9任一项所述的方法，其特征在于，所述待译码点云是待编码点云，若所述待处理边界像素块的类型对应多种处理方式；所述方法还包括：
    将标识信息编入码流，所述标识信息表示所述待处理边界像素块的目标处理方式。
  17. 根据权利要求6至9任一项所述的方法，其特征在于，所述待译码点云是待解码点云，若所述待处理边界像素块的类型对应多种处理方式，所述采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块，包括：
    根据所述待处理边界像素块的类型，解析码流，以得到标识信息；所述标识信息表示所述目标处理方式；
    采用所述标识信息所指示的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  18. 根据权利要求1至15任一项或权利要求17所述的方法，其特征在于，所述待译码点云是待解码点云，所述方法还包括：
    解析码流，以得到所述待译码点云的待处理边界像素块的尺寸信息；
    根据所述尺寸信息对待解码点云的占用图进行划分，得到一个或多个待处理边界像素块。
  19. 一种点云编码方法，其特征在于，包括：
    确定指示信息，所述指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；所述目标编码方法包括如权利要求1～16任一项所述的点云译码方法；
    将所述指示信息编入码流。
  20. 一种点云解码方法，其特征在于，包括：
    解析码流，以得到指示信息，所述指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；所述目标解码方法包括如权利要求1～15任一项或17或18所述的点云译码方法；
    当所述指示信息用于指示按照所述目标解码方法对所述待解码点云的占用图进行处理时，按照所述目标解码方法对所述待解码点云的占用图进行处理。
  21. 一种译码器，其特征在于，包括：
    分类模块，基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型；其中，所述无效像素块包括所包含的像素的值不全为0且不属于目标点云块patch占用图的像素块；或者，所述无效像素块包括所包含的像素的值全为0的像素块，和所包含的像素的值不全为0且不属于目标patch占用图的像素块；所述目标patch占用图是所述待处理边界像素块所属的patch占用图；
    重构模块，根据所述待处理边界像素块的类型，重构所述待译码点云。
  22. 根据权利要求21所述的译码器，其特征在于，所述分类模块还用于，当所述待译码点云的占用图的当前边界像素块的空域相邻像素块中的无效像素块的个数大于或等于预设阈值时，确定所述当前边界像素块为所述待处理边界像素块。
  23. 根据权利要求21或22所述的译码器，其特征在于，所述分类模块在所述基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型的方面，具体用于：基于所述待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的方位信息；其中，不同类型的边界像素块对应不同的方位信息。
  24. 根据权利要求23所述的译码器，其特征在于，如果所述待处理边界像素块的预设方位的空域相邻像素块是无效像素块，则确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的所述预设方位；其中，所述预设方位是正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方中的其中一种或者至少两种的组合。
  25. 根据权利要求21至24任一项所述的译码器，其特征在于，所述重构模块具体用于：根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块；根据经处理过的占用图，重构所述待译码点云，所述经处理过的占用图包括所述经目标操作的像素块；
    其中，所述重构模块在根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块的方面，具体用于：
    当所述目标操作是置0操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式将所述待处理边界像素块的目标位置的像素的值置0，得到经所述置0的像素块，不同处理方式对应的目标位置不同；
    或者，当所述目标操作是置1操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式将所述待处理边界像素块的目标位置的像素的值置1，得到经所述置1的像素块，不同处理方式对应的目标位置不同；
    或者，当所述目标操作是膨胀操作时，根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行膨胀操作，得到经膨胀操作后的像素块，不同处理方式的卷积核的半径不同。
  26. 根据权利要求25所述的译码器，其特征在于，所述重构模块在根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块的方面，具体用于：
    根据边界像素块的多种类型与多种处理方式之间的映射关系，确定所述待处理边界像素块的类型对应的处理方式；
    若所述待处理边界像素块的类型对应一种处理方式，则将所述待处理边界像素块的类型对应的处理方式作为所述目标处理方式；或者，若所述待处理边界像素块的类型对应多种处理方式，则将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式；
    采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  27. 根据权利要求25所述的译码器，其特征在于，所述重构模块在根据所述待处理边界像素块的类型，采用对应的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块的方面，具体用于：
    根据待处理边界像素块的类型查表，得到所述待处理边界像素块的类型对应的处理方式，所述表包括边界像素块的多种类型与多种处理方式之间的映射关系；
    若所述待处理边界像素块的类型对应一种处理方式，则将所述待处理边界像素块的类型对应的处理方式作为所述目标处理方式；或者，若所述待处理边界像素块的类型对应多种处理方式，则将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式；
    采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  28. 根据权利要求26或27所述的译码器，其特征在于，所述重构模块在所述将所述待处理边界像素块的类型对应的多种处理方式的其中一种处理方式作为所述目标处理方式的方面，具体用于：
    根据所述待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从所述待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为所述目标处理方式；所述目标空域相邻像素块是所述待处理边界像素块的为无效像素块的空域相邻像素块。
  29. 根据权利要求28所述的译码器，其特征在于，所述重构模块在所述根据所述待处理边界像素块的目标空域相邻像素块是否是空无效像素块，从所述待处理边界像素块的类型对应的多种处理方式中选择一种处理方式作为所述目标处理方式的方面，具体用于：
    如果所述目标空域相邻像素块是空无效像素块，则从所述待处理边界像素块的类型对应的多种处理方式中选择第一处理方式作为所述目标处理方式；
    如果所述目标空域相邻像素块是非空无效像素块，则从所述待处理边界像素块的 类型对应的多种处理方式中选择第二处理方式作为所述目标处理方式；
    其中，当所述目标操作是置0操作时，所述第一处理方式对应的目标位置的范围大于所述第二处理方式对应的目标位置的范围；
    或者，当所述目标操作是置1操作时，所述第一处理方式对应的目标位置的范围小于所述第二处理方式对应的目标位置的范围；
    或者，当所述目标操作是膨胀操作时，所述第一处理方式对应的卷积核的半径小于所述第二处理方式对应的卷积核的半径。
  30. 根据权利要求23或24所述的译码器，其特征在于，所述待处理边界像素块的空域相邻像素块包括：与所述待处理边界像素块相邻且位于所述待处理边界像素块的正上方、正下方、正左方和正右方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括正上方、正下方、正左方和正右方中的其中一种或至少两种的组合；
    或者，若所述待处理边界像素块的正上方和正右方的像素块为无效像素块，且所述待处理边界像素块的正下方和正左方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的右上方；
    或者，若所述待处理边界像素块的正下方和正左方的像素块为无效像素块，且所述待处理边界像素块的正上方和正右方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的左下方；
    或者，若所述待处理边界像素块的正上方和正左方的像素块为无效像素块，且所述待处理边界像素块的正下方和正右方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的左上方；
    或者，若所述待处理边界像素块的正下方和正右方的像素块为无效像素块，且所述待处理边界像素块的正上方和正左方的像素块是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的右下方。
  31. 根据权利要求23或24所述的译码器，其特征在于，所述待处理边界像素块的空域相邻像素块包括与所述待处理边界像素块相邻的且位于所述待处理边界像素块的左上方、右上方、左下方和右下方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括左上方、右上方、左下方和右下方其中一种或至少两种。
  32. 根据权利要求23或24所述的译码器，其特征在于，所述待处理边界像素块的空域相邻像素块包括：与所述待处理边界像素块相邻的且位于所述待处理边界像素块的正上方、正下方、正左方、正右方、左上方、右上方、左下方和右下方的像素块；
    若所述待处理边界像素块的预设方向的空域相邻像素块是无效像素块，且其他空域相邻像素块均是有效像素块，则所述方位信息是：所述待处理边界像素块中的无效像素位于所述待处理边界像素块中的所述预设方向；所述预设方向包括左上方、右上 方、左下方或右下方。
  33. 根据权利要求25至29任一项所述的译码器，其特征在于，
    当所述目标操作是置0操作时，所述目标位置是所述待处理边界像素块中的，与目标有效像素之间的距离大于或等于预设阈值的无效像素所在的位置；或者，所述目标位置是所述待处理边界像素块中的，且与目标有效像素所在的直线之间的距离大于或等于预设阈值的无效像素所在的位置；所述直线与所述待处理边界像素块的类型相关；
    或者，当所述目标操作是置1操作时，所述目标位置是所述待处理边界像素块中的，与目标有效像素之间的距离小于或等于预设阈值的无效像素所在的位置；或者，所述目标位置是所述待处理边界像素块中的，且与目标有效像素所在的直线之间的距离小于或等于预设阈值的无效像素所在的位置；所述直线与所述待处理边界像素块的类型相关。
  34. 根据权利要求21、22、24～33任一项所述的译码器，其特征在于，所述分类模块在所述基于待译码点云的占用图的待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块的类型的方面，具体用于：
    基于所述待处理边界像素块的空域相邻像素块是否是无效像素块，确定所述待处理边界像素块中的无效像素在所述待处理边界像素块中的方位信息；
    基于所述待处理边界像素块的为无效像素块的空域相邻像素块是否是空无效像素块，以及所述确定的方位信息，确定所述待处理边界像素块的类型。
  35. 根据权利要求34所述的译码器，其特征在于，所述待译码点云的占用图包括第一待处理边界像素块和第二待处理边界像素块；所述第一待处理边界像素块的类型是第一类型，所述第二待处理边界像素块的类型是第二类型；所述第一类型与所述第二类型所对应的方位信息相同；且所述第一待处理边界像素块的空域相邻像素块中预设方位的无效像素块是空无效像素块，所述第二待处理边界图像的空域相邻像素块中所述预设方位的无效像素块是非空无效像素块。
  36. 根据权利要求26至29任一项所述的译码器，其特征在于，所述译码器是编码器，若所述待处理边界像素块的类型对应多种处理方式；所述编码器还包括：
    辅助信息编码模块，用于将标识信息编入码流，所述标识信息表示所述待处理边界像素块的目标处理方式。
  37. 根据权利要求26至29任一项所述的译码器，其特征在于，所述译码器是解码器，若所述待处理边界像素块的类型对应多种处理方式，所述重构模块在所述采用所述目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块的方面，具体用于：
    根据所述待处理边界像素块的类型，解析码流，以得到标识信息；所述标识信息表示所述目标处理方式；
    采用所述标识信息所指示的目标处理方式对所述待处理边界像素块进行目标操作，得到经所述目标操作的像素块。
  38. 根据权利要求21至35任一项或权利要求37所述的译码器，其特征在于，所述译码器是解码器，所述解码器还包括：
    辅助信息解码模块，用于解析码流，以得到所述待译码点云的待处理边界像素块的尺寸信息；
    所述分类模块具体用于根据所述尺寸信息对待解码点云的占用图进行划分，得到一个或多个待处理边界像素块。
  39. 一种点云编码译码器，其特征在于，包括：
    辅助信息编码模块，用于确定指示信息，所述指示信息用于指示是否按照目标编码方法对待编码点云的占用图进行处理；所述目标编码方法包括如权利要求1～16任一项所述的点云译码方法；将所述指示信息编入码流。
  40. 一种点云解码译码器，其特征在于，包括：
    辅助信息解码模块，用于解析码流，以得到指示信息，所述指示信息用于指示是否按照目标解码方法对待解码点云的占用图进行处理；所述目标解码方法包括如权利要求要求1～15任一项或17或18所述的点云译码方法；当所述指示信息用于指示按照所述目标解码方法对所述待解码点云的占用图进行处理时，按照所述目标解码方法对所述待解码点云的占用图进行处理。
  41. 一种译码装置，其特征在于，包括存储器和处理器；所述存储器用于存储程序代码；所述处理器用于调用所述程序代码，以执行如权利要求1至18任一项所述的点云译码方法。
  42. 一种编码装置，其特征在于，包括存储器和处理器；所述存储器用于存储程序代码；所述处理器用于调用所述程序代码，以执行如权利要求19所述的点云编码方法。
  43. 一种解码装置，其特征在于，包括存储器和处理器；所述存储器用于存储程序代码；所述处理器用于调用所述程序代码，以执行如权利要求20所述的点云解码方法。
  44. 一种计算机可读存储介质，其特征在于，包括程序代码，所述程序代码在计算机上运行时，使得所述计算机执行如权利要求1至18任一项所述的点云译码方法。
  45. 一种计算机可读存储介质，其特征在于，包括程序代码，所述程序代码在计算机上运行时，使得所述计算机执行如权利要求19所述的点云编码方法。
  46. 一种计算机可读存储介质，其特征在于，包括程序代码，所述程序代码在计算机上运行时，使得所述计算机执行如权利要求20所述的点云解码方法。
PCT/CN2019/122485 2018-12-13 2019-12-02 点云编解码方法和编解码器 WO2020119509A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811527977.3A CN111327906B (zh) 2018-12-13 2018-12-13 点云编解码方法和编解码器
CN201811527977.3 2018-12-13

Publications (1)

Publication Number Publication Date
WO2020119509A1 true WO2020119509A1 (zh) 2020-06-18

Family

ID=71077109

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/122485 WO2020119509A1 (zh) 2018-12-13 2019-12-02 点云编解码方法和编解码器

Country Status (2)

Country Link
CN (1) CN111327906B (zh)
WO (1) WO2020119509A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113709093B (zh) * 2021-03-15 2023-08-04 上海交通大学 一种三维点云的封装方法、装置及介质
JP2024518154A (ja) * 2021-05-06 2024-04-25 オッポ広東移動通信有限公司 点群符号化・復号化方法、符号器、復号器及びコンピュータ記憶媒体

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104616278A (zh) * 2013-11-05 2015-05-13 北京三星通信技术研究有限公司 三维点云兴趣点检测方法和系统
US20170249401A1 (en) * 2016-02-26 2017-08-31 Nvidia Corporation Modeling point cloud data using hierarchies of gaussian mixture models
CN107301674A (zh) * 2017-05-22 2017-10-27 东南大学 一种基于窗口插值的三维重建纹理恢复方法
CN108053367A (zh) * 2017-12-08 2018-05-18 北京信息科技大学 一种基于rgb-d特征匹配的3d点云拼接与融合方法
US20180268570A1 (en) * 2017-03-16 2018-09-20 Samsung Electronics Co., Ltd. Point cloud and mesh compression using image/video codecs

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102074052A (zh) * 2011-01-20 2011-05-25 山东理工大学 基于样点拓扑近邻的散乱点云曲面拓扑重建方法
US9547901B2 (en) * 2013-11-05 2017-01-17 Samsung Electronics Co., Ltd. Method and apparatus for detecting point of interest (POI) in three-dimensional (3D) point clouds
CN105469447A (zh) * 2014-09-11 2016-04-06 富泰华工业（深圳）有限公司 点云边界直角边修补系统及方法
CN105574905B (zh) * 2015-12-15 2018-01-16 大连理工大学 一种三维激光点云数据的二维图像化表述方法
EP3642800A4 (en) * 2017-07-10 2020-05-20 Samsung Electronics Co., Ltd. POINT CLOUD AND NETWORK COMPRESSION WITH IMAGE / VIDEO CODECS
CN107463918B (zh) * 2017-08-17 2020-04-24 武汉大学 基于激光点云与影像数据融合的车道线提取方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104616278A (zh) * 2013-11-05 2015-05-13 北京三星通信技术研究有限公司 三维点云兴趣点检测方法和系统
US20170249401A1 (en) * 2016-02-26 2017-08-31 Nvidia Corporation Modeling point cloud data using hierarchies of gaussian mixture models
US20180268570A1 (en) * 2017-03-16 2018-09-20 Samsung Electronics Co., Ltd. Point cloud and mesh compression using image/video codecs
CN107301674A (zh) * 2017-05-22 2017-10-27 东南大学 一种基于窗口插值的三维重建纹理恢复方法
CN108053367A (zh) * 2017-12-08 2018-05-18 北京信息科技大学 一种基于rgb-d特征匹配的3d点云拼接与融合方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ROBERT A. COHEN ET AL: "Compression of 3-D point clouds using hierarchical patch fitting", 2017 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 20 September 2017 (2017-09-20), pages 4033 - 4037, XP033323333, DOI: 10.1109/ICIP.2017.8297040 *

Also Published As

Publication number Publication date
CN111327906B (zh) 2022-08-09
CN111327906A (zh) 2020-06-23

Similar Documents

Publication Publication Date Title
US11704837B2 (en) Point cloud encoding method, point cloud decoding method, encoder, and decoder
US20210029381A1 (en) Method and apparatus for obtaining global matched patch
CN110971898B (zh) 点云编解码方法和编解码器
WO2020011265A1 (zh) 点云编解码方法和编解码器
US11388442B2 (en) Point cloud encoding method, point cloud decoding method, encoder, and decoder
CN110944187B (zh) 点云编码方法和编码器
WO2020147379A1 (zh) 点云滤波方法、装置及存储介质
WO2020119509A1 (zh) 点云编解码方法和编解码器
CN111726615B (zh) 点云编解码方法及编解码器
WO2022134752A1 (en) Method and apparatus of entropy encoding/decoding point cloud geometry data captured by a spinning sensors head
WO2020063718A1 (zh) 点云编解码方法和编解码器
WO2020143725A1 (zh) 点云译码方法及译码器
KR20230169271A (ko) 스핀 센서 헤드에 의해 캡쳐된 포인트 클라우드 지오메트리 데이터에 대해 부호화/복호화하는 방법 및 장치(method and apparatus of encoding/decoding point cloud geometry data captured by a spinning sensors head)
WO2020187191A1 (zh) 点云编解码方法及编解码器
WO2020057338A1 (zh) 点云编码方法和编码器
WO2023050912A1 (en) Method and apparatus of encoding/decoding point cloud geometry data sensed by at least one sensor
WO2023040393A1 (en) Method and apparatus of encoding/decoding point cloud geometry datasensed by at least one sensor
KR20240027776A (ko) 스피닝 센서 헤드에 의해 캡처된 포인트 클라우드에 대해 인코딩/디코딩하는 방법 및 장치
KR20240065282A (ko) 적어도 하나의 센서에 의해 감지된 포인트 클라우드 지오메트리 데이터에 대해 인코딩/디코딩하는 방법 및 장치
KR20230162719A (ko) 방위각 코딩 모드를 사용하여 포인트 클라우드 지오메트리 데이터를 인코딩/디코딩하는 방법 및 장치(method and apparatus of encoding/decoding point cloud geometry data using azimuthal coding mode)

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19896534

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19896534

Country of ref document: EP

Kind code of ref document: A1