WO2022253228A1 - 3d地图的压缩、解压缩方法和装置 - Google Patents
3d地图的压缩、解压缩方法和装置 Download PDFInfo
- Publication number
- WO2022253228A1 WO2022253228A1 PCT/CN2022/096261 CN2022096261W WO2022253228A1 WO 2022253228 A1 WO2022253228 A1 WO 2022253228A1 CN 2022096261 W CN2022096261 W CN 2022096261W WO 2022253228 A1 WO2022253228 A1 WO 2022253228A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- descriptor
- representation
- substring
- map
- compact representation
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 195
- 238000007906 compression Methods 0.000 title claims abstract description 36
- 230000006835 compression Effects 0.000 title claims abstract description 35
- 230000006837 decompression Effects 0.000 title abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 105
- 238000005056 compaction Methods 0.000 claims abstract description 89
- 238000013507 mapping Methods 0.000 claims description 100
- 230000015654 memory Effects 0.000 claims description 61
- 230000005540 biological transmission Effects 0.000 claims description 28
- 238000013139 quantization Methods 0.000 claims description 24
- 238000005538 encapsulation Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 230000004044 response Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 7
- 238000013461 design Methods 0.000 description 89
- 230000000007 visual effect Effects 0.000 description 54
- 230000006870 function Effects 0.000 description 34
- 238000010586 diagram Methods 0.000 description 24
- 238000004891 communication Methods 0.000 description 21
- 230000008859 change Effects 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 10
- 238000013528 artificial neural network Methods 0.000 description 7
- 238000004806 packaging method and process Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 5
- 229920001621 AMOLED Polymers 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000002441 reversible effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000002096 quantum dot Substances 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
Definitions
- the present application relates to 3D map technology, in particular to a 3D map compression and decompression method and device.
- Virtual reality (virtual reality, VR), augmented reality (augmented reality, AR) and mixed reality (mixed reality, MR) technologies are emerging multimedia virtual scene technologies in recent years. This type of technology can create virtual reality and superimpose virtual reality and real world to generate new visual environment and interactive experience. In such applications, electronic devices need to determine their own pose information in the current environment, so as to accurately realize the fusion of virtual objects and real scenes.
- an embodiment of the present application provides a method for compressing a 3D map.
- the method may include: performing compaction processing on a descriptor to be encoded to obtain a first compacted representation of the descriptor to be encoded.
- the descriptor to be encoded corresponds to at least one 3D map point in the 3D map.
- the at least one reference descriptor corresponds to at least one encoded (or compressed) 3D map point of the 3D map.
- the data distribution of the first compact representation is different from the data distribution of the second compact representation.
- the first compacted representation of the descriptor to be encoded includes at least one first compacted representation substring of the descriptor to be encoded
- the second compacted representation of the descriptor to be encoded includes at least one substring of the descriptor to be encoded.
- a second compaction represents the substring. At least one first compacted representation substring is different from at least one second compacted representation substring.
- At least one first compact representation substring is different from the corresponding second compact representation substring respectively.
- the first compact representation includes 3 first compact representation substrings (A1, A2, and A3)
- the second compact representation includes 3 second compact representation substrings (B1, B2, and B3), wherein, A1 is different from B1, A2 is different from B2, and A3 is different from B3.
- part or all of at least one second compact representation substring is smaller than or equal to the corresponding first compact representation substring. Or, part or all of at least one second compact representation substring is greater than or equal to the corresponding first compact representation substring.
- the first table includes at least one encoded compact representation substring and at least one third compact representation substring corresponding to at least one second reference compact representation substring.
- the first compacted representation of the descriptor to be encoded comprises at least one first compacted representation substring of the descriptor to be encoded.
- the at least one encoded compact representation substring comprises at least one first compact representation substring
- the at least one second reference compact representation substring comprises at least one first reference compact representation substring of the at least one reference descriptor.
- the first table is a multidimensional array, and each element in the multidimensional array corresponds to at least one coded compact representation substring or at least one second reference compact representation substring.
- obtaining the second compact representation of the descriptor to be coded may include: converting at least one reference descriptor The XOR value or the difference between the compacted representation of and the first compacted representation of the descriptor to be coded is used as the second compacted representation of the descriptor to be coded.
- the embodiment of the present application provides a method for decompressing a 3D map.
- the method may include: decapsulating a code stream of the 3D map to obtain a second compact representation of the descriptor to be decoded.
- the descriptor to be decoded corresponds to at least one 3D map point in the 3D map.
- the at least one reference descriptor corresponds to at least one decoded (or decompressed) 3D map point of the 3D map.
- a first compacted representation of the descriptor to be decoded is obtained.
- reconstructed data of at least one 3D map point is obtained.
- the above method may further include: sending 3D map request information, and receiving a code stream of a 3D map corresponding to the 3D map request information. Or, receive the code stream of the 3D map sent by the electronic device.
- the first compacted representation of the descriptor to be decoded includes at least one first compacted representation substring of the descriptor to be decoded
- the second compacted representation of the descriptor to be decoded includes at least one substring of the descriptor to be decoded.
- a second compaction represents the substring. At least one first compacted representation substring is different from at least one second compacted representation substring.
- the first compacted representation of the descriptor to be decoded is obtained according to the second compacted representation of the descriptor to be decoded and the compacted representation of at least one reference descriptor corresponding to the descriptor to be decoded, including: according to For the second compact representation of the descriptor to be decoded, the first compact representation of the descriptor to be decoded is obtained by looking up the second table. The second table is established based on the compact representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the above device may further include: a transmission module.
- the transmission module is used to receive the 3D map request information sent by the electronic device, and in response to the 3D map request information, send the code stream of the 3D map corresponding to the 3D map request information to the electronic device, or the transmission module is used to send the 3D map to the server code stream.
- the first compacted representation of the descriptor to be encoded includes at least one first compacted representation substring of the descriptor to be encoded.
- the compact representation of the at least one reference descriptor comprises at least one first reference compact representation substring.
- the second table includes at least one first reference compact representation substring and at least one fourth compact representation substring corresponding to the at least one first compact representation substring.
- the mapping module is configured to: obtain at least one fourth compact representation substring corresponding to the at least one first compact representation substring from the second table according to at least one first compact representation substring of the descriptor to be encoded.
- the second table is built based on at least one first reference compact representation substring. According to at least one fourth compacted representation substring corresponding to the at least one first compacted representation substring, a second compacted representation of the descriptor to be encoded is obtained.
- the compaction processing includes quantization processing or binarization processing, and the quantization processing or binarization processing is used to reduce the number of bits of the descriptor to be encoded.
- the decapsulation module is configured to decapsulate the code stream of the 3D map to obtain a second compact representation of the descriptor to be decoded; the descriptor to be decoded corresponds to at least one 3D map point in the 3D map.
- An inverse mapping module configured to obtain a compact representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the at least one reference descriptor corresponds to at least one decoded 3D map point of the 3D map.
- the first compacted representation of the descriptor to be decoded includes at least one first compacted representation substring of the descriptor to be decoded
- the second compacted representation of the descriptor to be decoded includes at least one substring of the descriptor to be decoded.
- a second compaction represents the substring. At least one first compacted representation substring is different from at least one second compacted representation substring.
- part or all of at least one second compact representation substring is less than or equal to the corresponding first compact representation substring; or, part or all of at least one second compact representation substring is greater than or equal to Equal to the corresponding first compact representation substring.
- the anti-mapping module is used to: according to the second compact representation of the descriptor to be decoded and the compact representation of at least one reference descriptor corresponding to the descriptor to be decoded, by looking up the first table, obtain the The first compact representation of the descriptor.
- the demapping module is configured to obtain the first compacted representation of the descriptor to be decoded by looking up the second table according to the second compacted representation of the descriptor to be decoded.
- the second table is established based on the compact representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the inverse mapping module is used to: use the XOR value or sum value of at least one compacted representation of the reference descriptor and the second compacted representation of the descriptor to be decoded as the first A compact representation.
- the embodiment of the present application provides a 3D map compression device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are used by the one or more A plurality of processors are executed, so that the one or more processors implement the method described in the first aspect or any possible design of the first aspect.
- the embodiment of the present application provides a device for decompressing a 3D map, including: one or more processors; a memory for storing one or more programs; when the one or more programs are used by the one or multiple processors, so that the one or multiple processors implement the method described in the second aspect or any possible design of the second aspect.
- the embodiment of the present application provides a computer-readable storage medium, which is characterized in that it includes a computer program, and when the computer program is executed on a computer, the computer executes the first aspect or any one of the first aspect.
- the present application provides a computer program or computer program product, which, when the computer program or computer program product is executed on a computer, enables the computer to realize the above-mentioned first to second aspects and any possible implementation manners thereof. described method.
- the present application provides a non-transitory storage medium, including a bit stream encoded according to the method described in any one of the above-mentioned first aspects.
- the embodiment of the present application provides a method for compressing a 3D map.
- the method may include: performing clustering processing on the descriptors to be encoded to obtain at least one first center information of the descriptors to be encoded.
- the descriptor to be encoded corresponds to at least one 3D map point in the 3D map.
- the at least one first center information is used to indicate the cluster center of at least one sub-feature of the descriptor to be encoded.
- the reference descriptor corresponds to at least one encoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one second center information of the descriptor to be encoded is obtained according to at least one first center information and at least one first reference center information of the descriptor to be encoded.
- Perform compaction processing on at least one second central information of the descriptor to be coded to obtain a compact representation of the descriptor to be coded.
- the compact representation is encapsulated to obtain the code stream of the 3D map.
- the first center information is used to represent the original descriptor to be encoded, and the data volume of the descriptor to be encoded is reduced, and then at least one first center information is mapped to at least one second center information, so as to change the data distribution of the central information, and further reduce the data volume of the code stream of the 3D map obtained through compaction processing and packaging, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the descriptor to be encoded may be divided into one or more sub-features, and each sub-feature is clustered to obtain one or more first central information.
- the one or more first center information may be numbers of one or more cluster centers.
- the descriptor to be encoded may be an area descriptor or a 3D map point descriptor in a 3D map.
- the descriptor to be encoded may also be called the current descriptor.
- the reference descriptor corresponds to at least one encoded 3D map point of the 3D map.
- the reference descriptor may be an encoded 3D map point descriptor of the 3D map, or may be an encoded area descriptor of the 3D map.
- the reference descriptor corresponding to the descriptor to be encoded may be a reference area descriptor.
- the reference descriptor corresponding to the descriptor to be encoded may be a reference 3D map point descriptor.
- the method may further include: receiving 3D map request information sent by the electronic device, and in response to the 3D map request information, sending a code stream of a 3D map corresponding to the 3D map request information to the electronic device.
- the execution subject (which may be an electronic device or a server (such as a cloud server)) for executing the compression method described in the tenth aspect may receive the 3D map request information sent by other electronic devices, and send the 3D map request information to the electronic device.
- the code stream of the 3D map corresponding to the map request information may be an electronic device or a server (such as a cloud server) for executing the compression method described in the tenth aspect.
- the method may further include: sending the code stream of the 3D map to the server.
- the execution subject (which may be the first electronic device) for executing the compression method described in the tenth aspect may send the code stream of the 3D map to the server, and the server may send the code stream of the 3D map to the second electronic device.
- obtaining at least one second center information of the descriptor to be encoded according to at least one first center information and at least one first reference center information of the descriptor to be encoded may include: according to the At least one first central information of the descriptor and at least one first reference central information are obtained by looking up the first table to obtain at least one second central information of the descriptor to be encoded.
- the first table includes at least one coding center information and at least one third center information corresponding to at least one second reference center information.
- the at least one encoding center information includes at least one first center information of the descriptor to be encoded
- the at least one second reference center information includes the at least one first reference center information.
- at least one second center information of the descriptor to be encoded is obtained by looking up the first table, including: according to the descriptor to be encoded at least one first center information and at least one first reference center information, and at least one third center information corresponding to the at least one first center information and the at least one first reference center information is obtained from the first table.
- the at least one third central information is used as at least one second central information of the descriptor to be encoded.
- the first table is a multidimensional array, and each element in the multidimensional array corresponds to at least one coding center information or at least one second reference center information.
- obtaining at least one second central information of the descriptor to be encoded according to at least one first central information and at least one first reference central information of the descriptor to be encoded includes: according to the description to be encoded At least one first central information of the descriptor is obtained by looking up the second table to obtain at least one second central information of the descriptor to be encoded. The second table is established based on the at least one first reference center information.
- the second table includes the at least one first reference center information and at least one fourth center information corresponding to the at least one first center information of the descriptor to be encoded.
- at least one first central information of the descriptor to be encoded by looking up a second table, at least one second central information of the descriptor to be encoded is obtained, including: according to at least one first central information of the descriptor to be encoded, From the second table, at least one piece of fourth central information corresponding to the at least one first central information is acquired.
- the at least one fourth central information corresponding to the at least one first central information is used as at least one second central information of the descriptor to be encoded.
- the second table is an at least one-dimensional array, and each element in the at least one-dimensional array corresponds to at least one piece of first central information.
- obtaining at least one second central information of the descriptor to be encoded according to at least one first central information and at least one first reference central information of the descriptor to be encoded includes: An XOR value or a difference value of at least one first central information of the descriptor and at least one first reference central information is used as at least one second central information of the descriptor to be encoded.
- the embodiment of the present application provides a method for decompressing a 3D map.
- the method may include: decapsulating a code stream of the 3D map to obtain at least one second central information of the descriptor to be decoded.
- the descriptor to be decoded corresponds to at least one 3D map point in the 3D map.
- the reference descriptor corresponds to at least one decoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one first center information of the descriptor to be decoded is obtained according to at least one second center information of the descriptor to be decoded and at least one first reference center information of a reference descriptor corresponding to the descriptor to be decoded. Obtain reconstruction data of the at least one 3D map point according to at least one first center information of the descriptor to be decoded.
- the above method may further include: sending 3D map request information, and receiving a code stream of a 3D map corresponding to the 3D map request information. Or, receive the code stream of the 3D map sent by the electronic device.
- At least one second center information of the descriptor to be decoded and at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded at least one first center information of the descriptor to be decoded is obtained.
- a center information may include: according to at least one second center information of the descriptor to be decoded and at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded, by looking up the first table, the descriptor of the descriptor to be decoded is obtained At least one first center information.
- At least one second center information of the descriptor to be decoded and at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded at least one first center information of the descriptor to be decoded is obtained.
- a center information includes: according to at least one second center information of the descriptor to be decoded, at least one first center information of the descriptor to be decoded is obtained by looking up a second table. The second table is established based on at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded.
- At least one second center information of the descriptor to be decoded and at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded at least one first center information of the descriptor to be decoded is obtained.
- a center information including: an XOR value or a sum value of at least one second center information of the descriptor to be decoded and at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded, as the value to be decoded At least one first central information of the descriptor.
- the embodiment of the present application provides a device for compressing 3D maps.
- the device may be a chip or a system-on-chip in an electronic device or server, and may also be used in an electronic device or server to implement the tenth aspect or the tenth aspect.
- the apparatus may include: a mapping module, a compact processing module and a packaging module.
- the mapping module is configured to cluster the descriptors to be coded to obtain at least one first center information of the descriptors to be coded.
- the descriptor to be encoded corresponds to at least one 3D map point in the 3D map.
- the at least one first center information is used to indicate the cluster center of at least one sub-feature of the descriptor to be encoded.
- the mapping module is further configured to acquire at least one first reference center information of the reference descriptor corresponding to the descriptor to be encoded.
- the reference descriptor corresponds to at least one encoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one second center information of the descriptor to be encoded is obtained according to at least one first center information and at least one first reference center information of the descriptor to be encoded.
- the compaction processing module is configured to perform compaction processing on at least one second central information of the descriptor to be coded to obtain a compact representation of the descriptor to be coded.
- the encapsulation module is configured to encapsulate the compact representation to obtain a code stream of the 3D map.
- the above device may further include: a transmission module.
- the transmission module is used to receive the 3D map request information sent by the electronic device, and in response to the 3D map request information, send the code stream of the 3D map corresponding to the 3D map request information to the electronic device, or the transmission module is used to send the 3D map to the server code stream.
- the mapping module is specifically configured to: according to at least one first center information and at least one first reference center information of the descriptor to be coded, obtain at least one of the descriptors to be coded by looking up a first table Second center information.
- the first table is a multidimensional array, and each element in the multidimensional array corresponds to at least one coding center information or at least one second reference center information.
- the mapping module is specifically configured to: obtain at least one second central information of the descriptor to be encoded by looking up a second table according to at least one first central information of the descriptor to be encoded.
- the second table is established based on the at least one first reference center information.
- the second table includes the at least one first reference center information and at least one fourth center information corresponding to the at least one first center information of the descriptor to be encoded.
- the mapping module is specifically configured to: acquire at least one fourth central information corresponding to at least one first central information from the second table according to at least one first central information of the descriptor to be encoded.
- the at least one fourth central information corresponding to the at least one first central information is used as at least one second central information of the descriptor to be encoded.
- the mapping module is specifically configured to: use an exclusive OR value or a difference value of at least one first center information of the descriptor to be encoded and at least one first reference center information as at least A second central message.
- the embodiment of the present application provides a device for decompressing a 3D map.
- the device may be a chip or a system on a chip in an electronic device or a server, and may also be used in an electronic device or a server to implement the eleventh aspect or A functional module of the method described in any possible implementation manner of the eleventh aspect.
- the apparatus may include: a decapsulation module, a reverse mapping module and a reconstruction module.
- the demapping module is configured to acquire at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded.
- the reference descriptor corresponds to at least one decoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one first center information of the descriptor to be decoded is obtained according to at least one second center information of the descriptor to be decoded and at least one first reference center information of a reference descriptor corresponding to the descriptor to be decoded.
- the above device may further include: a transmission module, configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information. Or, it is used to receive the code stream of the 3D map sent by the electronic device.
- a transmission module configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information. Or, it is used to receive the code stream of the 3D map sent by the electronic device.
- the inverse mapping module is specifically configured to: according to at least one second central information of the descriptor to be decoded and at least one first reference central information of the reference descriptor corresponding to the descriptor to be decoded, by searching the first table to obtain at least one first center information of the descriptor to be decoded.
- the demapping module is specifically configured to: obtain at least one first central information of the descriptor to be decoded by looking up a second table according to at least one second central information of the descriptor to be decoded.
- the second table is established based on at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded.
- the inverse mapping module is specifically configured to: XOR the at least one second central information of the descriptor to be decoded with at least one first reference central information of the reference descriptor corresponding to the descriptor to be decoded The value or the sum value is used as at least one first central information of the descriptor to be decoded.
- the embodiment of the present application provides a 3D map compression device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are used by the one or multiple processors, so that the one or multiple processors implement the method described in the tenth aspect or any possible design of the tenth aspect.
- the embodiment of the present application provides a device for decompressing a 3D map, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the Execution by one or more processors, so that the one or more processors implement the method according to the eleventh aspect or any possible design of the eleventh aspect.
- an embodiment of the present application provides a computer-readable storage medium, which is characterized by comprising a computer program, and when the computer program is executed on a computer, the computer executes any one of the tenth aspect or the tenth aspect.
- the present application provides a computer program or a computer program product, when the computer program or computer program product is executed on a computer, the computer can realize the above tenth to eleventh aspects and any possible implementation thereof The method described by the method.
- the present application provides a non-transitory storage medium, including a bit stream encoded according to the method described in any one of the above tenth aspects.
- the third to ninth aspects of the present application are consistent with the technical solutions of the first to second aspects of the present application, and the beneficial effects obtained in each aspect and the corresponding feasible implementation manners are similar, so details are not repeated here.
- the twelfth to eighteenth aspects of the present application are consistent with the technical solutions of the tenth to eleventh aspects of the present application, and the beneficial effects obtained by the various aspects and the corresponding feasible implementation modes are similar, and will not be repeated here.
- FIG. 1 is a schematic diagram of an application architecture provided by an embodiment of the present application
- FIGS. 4a-4f are schematic diagrams of an application scenario provided by the embodiment of the present application.
- FIG. 5 is a schematic flowchart of a method for compressing a 3D map provided in an embodiment of the present application
- Fig. 6 is a schematic diagram comparing the data distribution of the second compact representation and the data distribution of the first compact representation provided by the embodiment of the present application;
- FIG. 7 is a schematic flowchart of a method for decompressing a 3D map provided in an embodiment of the present application.
- FIG. 10 is a schematic flowchart of a method for compressing a 3D map provided in an embodiment of the present application.
- FIG. 12 is a schematic flowchart of a method for compressing a 3D map provided in an embodiment of the present application.
- FIG. 15 is a schematic structural diagram of a device for decompressing a 3D map provided in an embodiment of the present application.
- FIG. 16 is a schematic block diagram of a decoding device 1600 provided by an embodiment of the present application.
- Fig. 17 is a schematic diagram of division of a first compact representation substring provided by the embodiment of the present application.
- FIG. 18 is a schematic flowchart of a method for compressing a 3D map provided by an embodiment of the present application.
- FIG. 19 is a schematic flowchart of a method for decompressing a 3D map provided by an embodiment of the present application.
- Fig. 1 is a schematic diagram of an application architecture provided by an embodiment of the present application.
- the application architecture includes a plurality of electronic devices and servers, wherein the plurality of electronic devices may include a first electronic device, one or more A second electronic device (two second electronic devices are taken as an example in FIG. 1), the one or more second electronic devices are several electronic devices other than the first electronic device. Multiple electronic devices and servers, and multiple electronic devices can communicate with each other.
- wireless-fidelity Wireless-fidelity
- WiFi wireless-fidelity
- Bluetooth Bluetooth
- cellular 2/3/4/5 generation 2/3/4/5generation, 2G/3G/4G/5G
- future communication methods which are not specifically limited.
- one or more second electronic devices in this embodiment of the present application is only used to refer to other electronic devices except the first electronic device, but does not limit whether the types of multiple electronic devices are the same.
- the above-mentioned servers may be one or more physical servers (a physical server is taken as an example in FIG. 1 ), may also be a computer cluster, or may be a virtual machine or a cloud server in a cloud computing scenario, and the like.
- the virtual scene application program in the electronic device may be an application program built in the electronic device itself, or an application program provided by a third-party service provider installed by the user himself, and no specific details are given here. limited.
- the collection device may include a visual information collection device in an electronic device and Inertial measurement unit (inertial measurement unit, IMU), wherein, the visual information acquisition device may include, for example, a camera, camera, depth camera, lidar, millimeter-wave radar, etc., and the IMU may include, for example, sensors such as a gyroscope and an accelerometer.
- IMU Inertial measurement unit
- the SLAM map is also referred to as a 3D map. It should be noted that the 3D map includes but is not limited to a SLAM map, and may also include a 3D map created using other technologies, which is not specifically limited in this embodiment of the present application.
- the spatial position of a 3D map point can be represented by X, Y, and Z on the axis of the three-dimensional space, it can also be represented by latitude and longitude, and altitude, and it can also be represented by polar coordinates.
- the embodiment of the present application expresses the spatial position of a 3D map point No specific restrictions are made.
- the spatial position of the 3D map point can be the absolute position of the 3D map point, or the relative position of the 3D map point. For example, taking the center position of the entire area as the origin, all the spatial positions of the 3D map point are relative to the spatial position of the origin offset position.
- each 3D map point can be assigned a number and written into the data of the 3D map, and the storage order of multiple 3D map points in the memory can also be used to implicitly represent the number of the 3D map point.
- the order of multiple 3D map points contained in a 3D map has no practical significance, so the aforementioned numbers can be considered as identifiers for identifying 3D map points to distinguish each 3D map point, but this Numbering is not used to limit the order of multiple 3D map points.
- a 3D map contains 3 3D map points, and their numbers are 1, 2, and 3 respectively.
- the processing of these 3 3D map points can be done according to The order of 1, 2, 3 can also be performed in the order of 3, 2, 1, and the order of 2, 1, 3 can also be performed, and so on.
- each 3D map point among multiple 3D map points can be described by some or all of the multiple region descriptors, so that 3D map points and region descriptors have a one-to-many relationship. It can be seen that there is a many-to-many relationship between multiple region descriptors and multiple 3D map points.
- the generation methods of regional descriptors include, but are not limited to, traditional methods such as bag of words (BOW), vector of locally aggregated descriptors (VLAD), and methods based on NetVLAD and artificial intelligence (AI). new method.
- BOW bag of words
- VLAD vector of locally aggregated descriptors
- AI artificial intelligence
- multiple region descriptors may also be identified with a number to distinguish the multiple region descriptors, but the number is also not a limitation on the order of the multiple region descriptors.
- the above correspondence can be explicitly described by a correspondence table between the number of the region descriptor and the number of the 3D map point.
- the 3D map contains 3 region descriptors, the numbers are T1 ⁇ T3, and there are 6 3D map points, where the numbers of the spatial positions of the six 3D map points are P 1 to P 6 , and the numbers of the six 3D map point descriptors are F 1 to F 6 , and the corresponding table is shown in Table 1.
- Table 1 is an example of the correspondence table between the number of the region descriptor and the number of the 3D map point, and the correspondence table can also be presented in other formats or ways, which is not specifically limited in this application.
- the electronic device 20 may include: a chip 21, a memory 22 (one or more computer-readable storage media), a user interface 23, a display component 24, a camera 25, a sensor 26, a positioning module 27 for positioning the device, and a communication device. Transceiver 28. Communication between these components may be via one or more buses 29 .
- the chip 21 may include: one or more processors 211 , a clock module 212 and a power management module 213 .
- the clock module 212 integrated in the chip 21 is mainly used to provide the processor 211 with a timer required for data transmission and timing control, and the timer can realize the clock function of data transmission and timing control.
- the processor 211 can perform operations according to instruction opcodes and timing signals, generate operation control signals, and complete control of fetching and executing instructions.
- the power management module 213 integrated in the chip 21 is mainly used to provide stable and high-precision voltage for the chip 21 and other components of the electronic device 20 .
- the processor 211 may include one or more interfaces.
- the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or, a universal serial bus (universal serial bus, USB) interface, etc.
- I2C integrated circuit
- I2S integrated circuit built-in audio
- PCM pulse code modulation
- PCM pulse code modulation
- UART universal asynchronous transmitter
- MIPI mobile industry processor interface
- GPIO general-purpose input and output
- subscriber identity module subscriber identity module
- SIM subscriber identity module
- USB universal serial bus
- the memory 22 may also store computer-executable program codes, where the computer-executable program codes include instructions, for example, communication program instructions, related program instructions of the SLAM system, and the like.
- the memory 22 can also store one or more application programs, for example, virtual scene application programs such as AR/VR/MR, map application programs, image management application programs, navigation and control application programs, and the like.
- the memory 22 can also store a user interface program.
- the user interface program can display the content of the application program through a graphical operation interface, for example, virtual objects in virtual scenes such as AR/VR/MR. 24 presentation, and realize receiving the user's control operations on the application through input controls such as menus, dialog boxes, and buttons.
- the user interface 23 may be, for example, a touch panel, and the touch panel may detect a user's operation instruction thereon, and the user interface 23 may also be, for example, a keypad, a physical button, or a mouse.
- Electronic device 20 may include one or more display components 24 .
- the electronic device 20 may jointly implement a display function through a display component 24 , a graphics processing unit (GPU) and an application processor (AP) in the chip 21 .
- the GPU is a microprocessor that implements image processing, which is connected to the display assembly 24 and the application processor, and the GPU performs mathematical and geometric calculations for graphics rendering.
- the display component 24 can display the interface content output by the electronic device 20, for example, display images, videos, etc. in virtual scenes such as AR/VR/MR, and the interface content can include the interface of the running application program and the system level menu, etc.
- interface elements such as buttons (Button), text input boxes (Text), sliders (Scroll Bar), menus (Menu), etc.; output interface elements, such as windows (Window), labels (Label), image, video, animation, etc.
- the display component 24 can be a display panel, glasses (such as VR glasses), projection screen and so on.
- the display panel may also be called a display screen, for example, may be a touch screen, a flexible screen, a curved screen, etc., or may be other optical components.
- the display screen of the electronic device in the embodiment of the present application may be a touch screen, flexible screen, curved screen or other forms of screen, that is, the display screen of the electronic device has the function of displaying images, and the specific The material and shape are not specifically limited.
- the display panel can be a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), an active matrix organic light-emitting diode or an active matrix Organic Light Emitting Diode (active-matrix organic light emitting diode, AMOLED), flexible light emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED )Wait.
- liquid crystal display liquid crystal display
- OLED organic light-emitting diode
- AMOLED active matrix organic light-emitting diode
- FLED flexible light emitting diode
- Miniled MicroLed, Micro-oLed
- quantum dot light emitting diodes quantum dot light emitting diodes (quantum dot light emitting diodes, QLED )Wait.
- the touch panel in the user interface 23 and the display panel in the display component 24 can be coupled together, for example, the touch panel can be arranged under the display panel, and the touch panel is used for Detecting the touch pressure on the display panel when the user inputs a touch operation (such as clicking, sliding, touching, etc.) through the display panel, and the display panel is used for displaying content.
- a touch operation such as clicking, sliding, touching, etc.
- the camera 25 may be a monocular camera, a binocular camera or a depth camera, and is used for shooting/recording the environment to obtain images/video images.
- the image/video image collected by the camera 25 can be used as a kind of input data of the SLAM system, or can be displayed by the display component 24 for image/video.
- the camera 25 can also be regarded as a sensor.
- the image collected by the camera 25 may be in the IMG format or in other formats, which is not specifically limited in this embodiment of the present application.
- the sensor 26 can be used to collect data related to state changes of the electronic device 20 (for example, rotation, swing, movement, shaking, etc.), and the data collected by the sensor 26 can be used as an input data of the SLAM system.
- the sensor 26 may include one or more sensors, for example, an inertial measurement unit (inertial measurement unit, IMU), a time of flight (time of flight, TOF) sensor, and the like.
- the IMU may include sensors such as a gyroscope and an accelerometer.
- the gyroscope is used to measure the angular velocity of the electronic device when it is in motion
- the accelerometer is used to measure the acceleration of the electronic device when it is in motion.
- the TOF sensor can include a light emitter and a light receiver.
- the light emitter is used to emit light, such as laser, infrared, radar waves, etc.
- the light receiver is used to detect reflected light, such as reflected laser, infrared, and radar waves. Wait.
- the sensor 26 may also include more other sensors, such as inertial sensors, barometers, magnetometers, wheel speedometers, etc., which are not specifically limited in this embodiment of the present application.
- the positioning module 27 is used to realize the physical positioning of the electronic device 20 , for example, to acquire the initial position of the electronic device 20 .
- the positioning module 27 may include one or more of a WiFi positioning module, a Bluetooth positioning module, a base station positioning module, and a satellite positioning module.
- the satellite positioning module can be provided with a global navigation satellite system (global navigation satellite system, GNSS) to assist positioning, and GNSS is not limited to the Beidou system, the global positioning system (global positioning system, GPS) system, the GLONASS (global navigation satellite system) , GLONASS) system, Galileo satellite navigation system (Galileo) system.
- the transceiver 28 is used to enable communication between the electronic device 20 and other devices (eg, servers, other electronic devices, etc.).
- the transceiver 28 integrates a transmitter and a receiver for sending and receiving radio frequency signals, respectively.
- the transceiver 28 includes, but is not limited to: an antenna system, a radio frequency (radio frequency, RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec ( CODEC) chip, subscriber identification module (SIM) card and storage media, etc.
- the transceiver 28 can also be implemented on a separate chip.
- the transceiver 28 supports at least one data network communication in 2G/3G/4G/5G, etc., and/or supports at least one of the following short-range wireless communication methods: Bluetooth (bluetooth, BT) communication, wireless fidelity (wireless fidelity, WiFi) communication, near field communication (near field communication, NFC), infrared (infrared, IR) wireless communication, ultra wide band (ultra wide band, UWB) communication, ZigBee protocol (ZigBee) communication.
- Bluetooth bluetooth, BT
- wireless fidelity wireless fidelity, WiFi
- NFC near field communication
- infrared infrared
- UWB ultra wide band
- ZigBee protocol ZigBee protocol
- the processor 211 executes various functional applications and data processing of the electronic device 20 by running the program code stored in the memory 22 .
- FIG. 3 is a schematic structural diagram of a server 30 provided in an embodiment of the present application.
- the server 30 may be the server in the embodiment shown in FIG. 1 .
- Server 30 includes processor 301 , memory 302 (one or more computer-readable storage media), and transceiver 303 . Communication between these components may be via one or more buses 304 .
- the memory 302 can be connected to the processor 301 through the bus 304, or can be coupled with the processor 301, and is used to store various program codes and/or multiple sets of instructions, and data (eg, map data, pose data, etc.).
- the memory 302 includes, but is not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM), or portable read-only memory (Compact Disc Read-Only Memory, CD-ROM), etc.
- the transceiver 303 mainly integrates a receiver and a transmitter, where the receiver is used to receive data sent by the electronic device (for example, requests, images, etc.), and the transmitter is used to send data to the electronic device (for example, map data, pose data, etc.) .
- server 30 shown in FIG. 3 is only an example provided by the embodiment of the present application, and the server 30 may have more components than shown in the figure, which is not specifically limited in the embodiment of the present application.
- Coupled used in the embodiments of the present application means to be directly connected or connected through one or more intermediate components or circuits.
- Fig. 4a is a schematic diagram of an application scenario provided by the embodiment of the present application.
- the application scenario is that the electronic device collects visual information through a sensor, and determines the current pose of the electronic device in combination with the visual information and the 3D map from the server.
- the 3D map is provided by the server, that is, the server creates the 3D map, then compresses the 3D map, and transmits the compressed data of the 3D map to the electronic device; the electronic device performs decompression processing after receiving the compressed data of the 3D map, and obtains
- the reconstruction data of the 3D map is combined with the collected visual information and the reconstruction data of the 3D map to determine the current pose of the electronic device.
- the pose is the position information of the electronic device, which can be the absolute pose in the world coordinate system, or the relative pose relative to a certain point in the environment.
- the server may pre-create the 3D map, compress the 3D map and store it locally, which can save storage space.
- the server can also transmit the compressed data of the 3D map to other devices, such as cloud storage.
- the server creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
- Compressing the 3D map by the server can save local storage space.
- the electronic device sends a map download request to the server. There are two ways to trigger the map download request:
- the user opens the map application program installed on the electronic device, and the application program uploads the location information obtained based on GPS positioning or WiFi positioning to its corresponding server, and the uploading operation can trigger a map download request. Since the uploaded content includes location information, the server can make a preliminary estimate based on the location information, and transmit the compressed data of the 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device.
- the scope of the area to which the anchor point indicated by the location information can be preset.
- the area to which the anchor point belongs can be the administrative region of each level (including counties, cities, countries, or administrative districts, etc.) where the anchor point is located, or it can be centered on the anchor point , Set the distance as the circle area of the radius.
- the user opens the map application program installed on the electronic device, and actively inputs or selects an area in the application program, for example, the user actively enters "xx business center", or selects from a list "A street, B street, "A Street” was selected in "C Street”.
- the foregoing operations of the user may trigger a map download request.
- Either the user input or selection includes a geographic location, so the server transmits the compressed data of the 3D map of the geographic location to the electronic device.
- the electronic device automatically detects whether it meets the conditions for downloading a 3D map or starts downloading a 3D map, or the electronic device passes the detection A change in ambient light or a change in the environment is detected to start downloading the 3D map, so as to request the server to download a 3D map within an area, and the size of the area is not specifically limited.
- the server sends the compressed data of the 3D map to the electronic device.
- Electronic equipment collects visual information through sensors.
- step 3 and step 4 are independent of each other, and the sequence is not limited.
- the electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
- the electronic device After receiving the compressed data of the 3D map, the electronic device does not need to decompress it immediately, but only needs to decompress to obtain the reconstruction data of the 3D map before positioning based on the visual information. For example, the user can pre-download the compressed data of a 3D map in an area by downloading an "offline map", and decompress the compressed data of the 3D map when positioning is required.
- Figure 4b is a schematic diagram of an application scenario provided by the embodiment of the present application.
- the application scenario is that the electronic device collects visual information through a sensor, and the server determines the current position of the electronic device in combination with the visual information from the electronic device and a 3D map. posture.
- the 3D map is provided by the server, that is, the server creates the 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally.
- the server performs decompression processing to obtain the reconstructed data of the 3D map, and combines the visual information and the reconstructed data of the 3D map to determine the current pose of the electronic device.
- the server creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
- Electronic devices collect visual information through sensors.
- the electronic device sends the visual information to the server.
- the server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the server compresses the 3D map to save storage space.
- the server performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
- the server sends the pose to the electronic device.
- Fig. 4c is a schematic diagram of an application scenario provided by the embodiment of the present application.
- the application scenario is that the electronic device collects visual information through a sensor, and determines the current pose of the electronic device in combination with the collected visual information and a 3D map.
- the 3D map is provided by the electronic device, that is, the electronic device creates the 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally.
- the electronic device performs decompression processing to obtain the reconstructed data of the 3D map, and combines the collected visual information and the reconstructed data of the 3D map to determine the current pose of the electronic device.
- the electronic device creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
- Electronic equipment collects visual information through sensors.
- the electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
- Fig. 4d is a schematic diagram of an application scenario provided by an embodiment of the present application.
- the application scenario is that the second electronic device collects visual information through a sensor, and determines the location of the second electronic device in combination with the visual information and the 3D map from the server. current pose.
- the 3D map is created by the first electronic device, that is, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed data of the 3D map to the server, and the server sends the compressed data of the 3D map to the second
- the second electronic device performs decompression processing to obtain the reconstruction data of the 3D map, and determines the current pose of the second electronic device in combination with the collected visual information and the reconstruction data of the 3D map.
- the first electronic device may pre-create a 3D map, compress the 3D map and then transmit it to the server, which can reduce transmission bandwidth.
- the first electronic device sends the compressed data of the 3D map to the server.
- the first electronic device compresses the 3D map before transmitting it, which can reduce the transmission bandwidth and improve the transmission efficiency.
- the second electronic device sends a map download request to the server.
- the map download request sent by the second electronic device may also be based on the trigger manner shown in FIG. 4a.
- the server sends the compressed data of the 3D map to the second electronic device.
- the second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the second electronic device collects the visual information through the sensor.
- the second electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
- Fig. 4e is a schematic diagram of an application scenario provided by the embodiment of the present application.
- the application scenario is that the second electronic device collects visual information through a sensor, and the server combines the visual information from the second electronic device with the visual information from the first electronic device.
- the 3D map of the device determines the current pose of the second electronic device.
- the 3D map is created by the first electronic device, that is, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed data of the 3D map to the server.
- the server performs decompression processing to obtain the reconstructed data of the 3D map, and combines the visual information from the second electronic device and the reconstructed data of the 3D map to determine the current pose of the second electronic device.
- the first electronic device creates a 3D map and compresses it to obtain compressed data of the 3D map.
- the first electronic device sends the compressed data of the 3D map to the server.
- the second electronic device acquires visual information through sensor collection.
- the second electronic device sends a positioning request to the server, and the positioning request carries visual information.
- the server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the server performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
- the server sends the pose obtained by positioning to the second electronic device.
- the 3D map is created by the first electronic device, that is, the first electronic device creates a 3D map, performs compression processing on the 3D map, and then sends the compressed data of the 3D map to the second electronic device, and the second electronic device performs decompression processing , obtain the reconstructed data of the 3D map, and determine the current pose of the second electronic device in combination with the collected visual information and the 3D map from the first electronic device.
- the first electronic device sends the compressed data of the 3D map to the second electronic device.
- the second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
- the positioning algorithm used may include:
- the reconstructed data of the 3D map obtained through decompression involved in the embodiment of the present application may also be referred to as the reconstructed data of the 3D map.
- the data distribution of the second compact representation is different from the data distribution of the first compact representation.
- the data distribution of the compact representation refers to the distribution of binary numbers in the compact representation.
- the data amount of the code stream of the 3D map obtained through subsequent encapsulation is reduced, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the first compacted representation may include at least one first compacted representation substring
- the second compacted representation may include at least one second compacted representation substring
- the compact representation of each reference descriptor in the at least one reference descriptor includes at least one first reference compact representation substring.
- FIG. 6 is a schematic diagram of comparing the data distribution of the second compact representation and the data distribution of the first compact representation provided by the embodiment of the present application. As shown in FIG. 6, the horizontal axis is the compact representation string, the vertical axis is the frequency of occurrence, the slanted bar is the data distribution of the first compact representation, and the blank bar is the data distribution of the second compact representation. It can be seen from Fig.
- the second compact representation obtained through the above compression processing of S101 to S103 may also be called compressed data.
- the compressed data is encapsulated through S104 to obtain the code stream of the 3D map.
- Such compression processing may also be referred to as encoding processing.
- the second compact representation of the descriptor to be decoded in this embodiment is the second compact representation of the descriptor to be encoded in the embodiment of FIG. 5 above.
- the second compact representation of the descriptor to be decoded is back-mapped to the first compact representation of the descriptor to be decoded.
- the first compacted representation of the descriptor to be decoded here is the first compacted representation of the descriptor to be encoded in the above embodiment in FIG. 5 .
- the second compact representation of the descriptor to be decoded is obtained, and according to the second compact representation of the descriptor to be decoded and at least one reference corresponding to the descriptor to be decoded
- the compacted representation of the descriptor is obtained by obtaining a first compacted representation of the descriptor to be decoded, and the reconstructed data of at least one 3D map point is obtained according to the first compacted representation of the descriptor to be decoded.
- the encoding end undergoes compaction processing first to reduce the data volume of the descriptors to be encoded, and then maps the first compact representation to the second compact representation to change the data distribution of the compact representation and further reduce the 3D representation obtained through encapsulation.
- the data volume of the code stream of the map thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the decoding end may perform inverse mapping on the second compact representation obtained by decapsulation to obtain the first compact representation, and then obtain reconstruction data of 3D map points.
- mapping of the first compact representation to the second compact representation may be implemented in different manners.
- Method 1 the first table mapping (also known as static table mapping)
- the second compacted representation of the descriptor to be coded is obtained by looking up the first table.
- the first table can be preset.
- the second compact representation of the descriptor to be coded is obtained by looking up the second table.
- the second table is built based on a compact representation of at least one reference descriptor.
- An XOR value or a difference between at least one compacted representation of the reference descriptor and the first compacted representation of the descriptor to be coded is used as a second compacted representation of the descriptor to be coded.
- FIG. 8 is a schematic flowchart of a method for compressing a 3D map provided by an embodiment of the present application.
- the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or can also be Applied to the server shown in any one of Fig. 1 to Fig. 4f.
- the method for compressing a 3D map in this embodiment adopts the above method 1 to map the first compact representation to the second compact representation.
- This embodiment is described by taking an example in which the first compact representation includes at least one first compact representation substring, and the second compact representation includes at least one second compact representation substring.
- the method includes but is not limited to the following steps:
- the first compacted representation includes at least one first compacted representation substring.
- the compact representation of the at least one reference descriptor comprises at least one first reference compact representation substring of each of the at least one reference descriptor.
- the first table in this embodiment may include at least one encoded compact representation substring and at least one third compact representation substring corresponding to at least one second reference compact representation substring.
- At least one coded compact representation substring of the first table includes at least one first compact representation substring of the descriptor to be encoded, and at least one second reference compact representation substring of the first table includes the at least one reference descriptor The at least one first reference compacts the representation substring.
- at least one corresponding third compacted representation substring may be acquired from the first table according to at least one first compacted representation substring and at least one first reference compacted representation substring.
- the first table can have many different forms.
- the first table may be a multidimensional array, for example, a two-dimensional array or a three-dimensional array.
- T[i][j] represents the third compacted representation substring corresponding to the coded compacted representation substring i and the second reference compacted representation substring j.
- i takes 0 to N-1 respectively
- j takes 0 to M-1 respectively.
- N and M are positive integers greater than 1.
- N represents the total number of coded compact representation substrings
- M represents the total number of second reference compact representation substrings.
- the first compaction representation substring when using the first table to map the first compaction representation substring, the first compaction representation substring can be used, and the N encoding compaction representation substrings , find the same substring x as the first compact representation substring, x ⁇ [0,N-1], use the first reference compact representation substring corresponding to the first compact representation substring, in M
- the second reference compact representation substring find the same substring y as the first reference compact representation substring, y ⁇ [0,M-1], T[x][y] is the corresponding first Three compact representations of substrings.
- T[i][j] can be obtained through function mapping.
- the function may be a function of the encoded compact representation substring i and the reference compact representation substring j.
- the function may be a random function.
- the function may also be other functions, and its specific form is not specifically limited in this embodiment of the present application.
- T[i][j] is less than or equal to the coded compact representation substring i, or greater than or equal to the coded compact representation substring i.
- T[i][j] of the first table is less than or equal to the coded compact representation substring i, then the third compact representation substring obtained through the first table is less than or equal to the first compact representation substring.
- T[i][j] of the first table is greater than or equal to the coded compact representation substring i, then the third compact representation substring obtained through the first table is greater than or equal to the first compact representation substring.
- T[i][j][1], T[i][j][2], ..., and T[i][j][K] Indicates the K third compacted representation substrings corresponding to the coded compacted representation substring i and the reference compacted representation substring j.
- i takes 0 to N-1 respectively
- j takes 0 to M-1 respectively.
- N, M, and K are positive integers greater than 1.
- N represents the total number of coded compact representation substrings
- M represents the total number of reference compact representation substrings.
- the first compaction representation substring when using the first table to map the first compaction representation substring, the first compaction representation substring can be used, and the N encoding compaction representation substrings , find the same substring x as the first compacted representation substring, x ⁇ [0,N-1], use the compacted representation substring corresponding to the first compacted representation substring, in M reference compact
- the compact representation substring find the same substring y as the compact representation substring, y ⁇ [0,M-1], T[x][y][1], T[x][y][2 ], ..., and T[x][y][K] are the corresponding plurality of third compact representation substrings.
- T[i][j][1], T[i][j][2], ..., and T[i][j][K] can be obtained through function mapping.
- the function may be a function of the encoded compact representation substring i and the reference compact representation substring j.
- the function may be a random function.
- the function may also be other functions, and its specific form is not specifically limited in this embodiment of the present application.
- T[i][j 1 ][j 2 ] represents the coded compact representation substring i, the second reference compact representation substring j 1 and the second reference compact representation substring j 2 corresponding to the first Three compact representations of substrings.
- i takes 0 to N-1 respectively
- j 1 and j 2 take 0 to M-1 respectively.
- N and M are positive integers greater than 1.
- N represents the total number of coded compact representation substrings
- M represents the total number of second reference compact representation substrings.
- the first compaction representation substring when using the first table to map the first compaction representation substring, the first compaction representation substring can be used, and the N encoding compaction representation substrings , find the same substring x as the first compacted representation substring, x ⁇ [0,N-1], use a second reference compacted representation substring j 1 corresponding to the first compacted representation substring , in the M second reference compact representation substrings, find the same substring y as the second compact representation substring j 1 , y ⁇ [0,M-1], use the first compact representation substring Another second reference compact representation substring j 2 corresponding to the string, in the M second reference compact representation substrings, find the same substring z as the second compact representation substring j 2 , z ⁇ [ 0,M-1], T[x][y][z] is the corresponding third compact representation substring.
- T[x][y][z] can be obtained through function mapping.
- the function may be a function on the encoded compact representation substring i, the second reference compact representation substring j1 and the second reference compact representation substring j2 .
- the function may be a random function.
- the function may also be other functions, and its specific form is not specifically limited in this embodiment of the present application.
- a first compact representation substring and a first reference compact representation substring correspond to a third compact representation substring.
- the obtained at least one second compacted representation substring is the second compacted representation of the descriptor to be encoded. Since T[i][j] in the first table is less than or equal to the coded compact representation substring i, the first compact representation substring is used, and the second compact representation substring obtained through the first table mapping is less than or equal to the first compacted representation substring.
- the first compact representation substring can be mapped toward the first direction, and the first direction can be a direction with a smaller numerical value (for example, the abscissa shown in FIG. 6 is to the left direction), so that the second compact representation substring has a greater probability of being a smaller value, so that the data distribution of the second compact representation is more concentrated, so as to reduce the data volume of the code stream of the 3D map obtained through subsequent entropy coding.
- T[i][j] in the first table is greater than or equal to the coded compact representation substring i, the first compact representation substring is used, and the second compact representation substring obtained through the mapping of the first table Greater than or equal to the first compact representation substring.
- the first compact representation substring can be mapped toward the second direction, and the second direction can be a direction with a larger value (for example, the abscissa shown in FIG. 6 is to the right direction), so that the second compact representation substring has a greater probability of being a larger value, so that the data distribution of the second compact representation is more concentrated, so as to reduce the data volume of the code stream of the 3D map obtained through subsequent entropy coding.
- a larger value for example, the abscissa shown in FIG. 6 is to the right direction
- one first compact representation substring and one first reference compact representation substring correspond to multiple third compact representation substrings.
- a second compacted representation substring is obtained according to the corresponding plurality of third compacted representation substrings. In this way, at least one second compact representation substring is obtained, which is the second compact representation of the descriptor to be encoded.
- a plurality of third compacted representation substrings corresponding to a first compacted representation substring and a first reference compacted representation substring are T[i][j][1], T[i][j ][2],..., and T[i][j][K], then according to T[i][j][1], T[i][j][2],..., and T [i][j][K] yields a second compact representation substring.
- T[i][j][1], T[i][j][2], ..., and T[i][j][K] are averaged, and the average value is used as a second compact Represents a substring.
- a second compact representation substring may also be obtained according to multiple third compact representation substrings in other manners, and this embodiment of the present application does not illustrate one by one. Since the second compact representation substring obtained through the first table is less than or equal to the encoded compact representation substring i, the first compact representation substring is used, and the second compact representation substring obtained through the first table mapping is less than or equal to the first compacted representation substring. Thus, through the first table mapping, the first compact representation substring can be mapped toward the first direction, and the first direction can be a direction with a smaller numerical value (for example, the abscissa shown in FIG.
- the second compact representation substring has a greater probability of being a smaller value, so that the data distribution of the second compact representation is more concentrated, so as to reduce the data volume of the code stream of the 3D map obtained through subsequent entropy coding.
- the second compact representation substring obtained through the first table is greater than or equal to the encoded compact representation substring i, the first compact representation substring is used, and the second compact representation substring obtained through the mapping of the first table string is greater than or equal to the first compacted representation substring.
- the first compact representation substring can be mapped toward the second direction, and the second direction can be a direction with a larger value (for example, the abscissa shown in FIG.
- the second compact representation substring has a greater probability of being a larger value, so that the data distribution of the second compact representation is more concentrated, so as to reduce the data volume of the code stream of the 3D map obtained through subsequent entropy coding.
- the order of at least one first compact representation substring is different from the order of at least one second compact representation substring.
- At least one first compacted representation substring of the descriptor to be coded is obtained, and according to at least one first compacted representation substring and at least one first reference compacted representation String, from the first table, obtain the at least one first compacted representation substring and at least one third compacted representation substring corresponding to the at least one first reference compacted representation substring, according to at least one first compacted representation substring represent the substring and at least one third compacted representation substring corresponding to at least one first reference compacted representation substring, obtain a second compacted representation of the descriptor to be encoded, and encapsulate the second compacted representation to obtain The code stream of the 3D map.
- the data volume of the descriptor to be encoded is reduced, and then the first compact representation is mapped to the second compact representation through the first table, so as to change the data distribution of the compact representation, and further reduce the encapsulation obtained
- the data volume of the code stream of the 3D map is reduced, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- FIG. 9 is a schematic flow chart of a method for decompressing a 3D map provided in an embodiment of the present application.
- the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
- the method for decompressing a 3D map in this embodiment uses the first table to demap the second compact representation into the first compact representation. This embodiment is described by taking an example in which the first compact representation includes at least one first compact representation substring, and the second compact representation includes at least one second compact representation substring.
- the method includes but is not limited to the following steps:
- S401 Decapsulate the code stream of the 3D map to obtain a second compact representation of the descriptor to be decoded.
- the second compacted representation includes at least one second compacted representation substring.
- the code stream of the 3D map may be the code stream of the 3D map obtained through the embodiment shown in FIG. 8 , or the code stream of the 3D map obtained through the embodiment shown in FIG. 10 .
- the compression method and decompression method in the embodiments of the present application can be flexibly combined.
- the compact representation of the at least one reference descriptor comprises at least one first reference compact representation substring of each of the at least one reference descriptor.
- the first table in this embodiment can be the same table as the first table in the embodiment shown in Figure 8, or it can be a table different from the first table in the embodiment shown in Figure 8, for example, it can be
- the first table in the illustrated embodiment corresponds to the inverse mapping table.
- the compact representation before mapping can be obtained through the first table, and then the reconstructed data of the 3D map points can be obtained.
- T[i][j] represents the third compact representation substring corresponding to the coding compact representation substring i and the second reference compact representation substring j
- the third compact representation substring The substring is represented by a second compaction as a substring.
- T'[i'][j] the reverse mapping table of this embodiment can be expressed as T'[i'][j], that is, the reverse mapping table is obtained
- At least one second compact representation substring of the descriptor to be decoded is obtained, and according to the at least one second compact representation substring of the descriptor to be decoded and at least one Refer to at least one first reference compacted representation substring of the descriptor, obtain at least one first compacted representation substring of the descriptor to be decoded from the first table, and obtain at least one first compacted representation substring of the descriptor to be decoded according to the at least one first compacted representation substring of the descriptor to be decoded denote the substring to obtain the reconstructed data of at least one 3D map point.
- the encoding end undergoes compaction processing first to reduce the data volume of the descriptors to be encoded, and then maps the first compact representation to the second compact representation to change the data distribution of the compact representation and further reduce the 3D representation obtained through encapsulation.
- the data volume of the code stream of the map thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the decoding end can use the first table to perform inverse mapping on the second compact representation obtained by decapsulation to obtain the first compact representation, and then obtain the reconstructed data of the 3D map points.
- FIG. 10 is a schematic flowchart of a method for compressing a 3D map provided by an embodiment of the present application.
- the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f , or can also be Applied to the server shown in any one of Fig. 1 to Fig. 4f.
- the method for compressing a 3D map in this embodiment uses the second method above to map the first compact representation to the second compact representation. This embodiment is described by taking an example in which the first compact representation includes at least one first compact representation substring, and the second compact representation includes at least one second compact representation substring.
- the method includes but is not limited to the following steps:
- S501 Perform compaction processing on the descriptor to be encoded to obtain a first compacted representation of the descriptor to be encoded.
- the first compacted representation includes at least one first compacted representation substring.
- the compact representation of the at least one reference descriptor comprises at least one first reference compact representation substring of each of the at least one reference descriptor.
- the second table is established based on a compact representation of at least one reference descriptor corresponding to the descriptor to be encoded.
- the second table in this embodiment may include the first reference compact representation substring and at least one fourth compact representation substring corresponding to at least one first compact representation substring string. That is, a first reference compacts the representation substring, a second table. A plurality of first reference compact representation substrings can be used to create a plurality of second tables.
- the second table can take many different forms.
- the second table may be a one-dimensional array or a multi-dimensional array
- the multi-dimensional array may be, for example, a two-dimensional array or a three-dimensional array.
- T[k] represents the fourth compacted representation substring corresponding to the first compacted representation substring k.
- k ranges from 0 to N-1 respectively
- N is the number of substrings represented by the first compaction
- N is a positive integer greater than 1
- k is the substring k of the first compacted representation.
- T[k] can be obtained through function mapping.
- the function may be a random function.
- the function may also be other functions, and its specific form is not specifically limited in this embodiment of the present application.
- T[k] is less than or equal to the first compacted representation substring k, or greater than or equal to the first compacted representation substring k.
- the fourth compact representation substring obtained through the second table is less than or equal to the first compact representation substring.
- T[k] of the first table is greater than or equal to the first compact representation substring k
- the fourth compact representation substring obtained through the second table is greater than or equal to the first compact representation substring
- At least one first compacted representation substring of the descriptor to be coded is obtained by compacting the descriptor to be coded, and the at least one compacted representation substring is obtained from the second table according to the at least one first compacted representation substring.
- a first compacted representation substring corresponds to at least one fourth compacted representation substring.
- the second table is built from at least one first reference compact representation substring of at least one reference descriptor.
- a second compacted representation of the descriptor to be encoded is obtained, and the second compacted representation is encapsulated to obtain a code stream of a 3D map.
- the data volume of the descriptor to be encoded is reduced, and then the first compact representation is mapped to the second compact representation through the second table, so as to change the data distribution of the compact representation, and further reduce the encapsulation obtained
- the data volume of the code stream of the 3D map is reduced, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- FIG. 11 is a schematic flowchart of a method for decompressing a 3D map provided by an embodiment of the present application.
- the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f , or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
- the method for decompressing a 3D map in this embodiment uses the second table to demap the second compact representation into the first compact representation.
- the first compact representation includes at least one first compact representation substring
- the second compact representation includes at least one second compact representation substring as an example for illustration.
- the method includes but is not limited to the following steps:
- the second table is built from at least one first reference compact representation substring of the one or more reference descriptors.
- N exclusive OR values or difference values are used as N second compact representation substrings.
- the N exclusive OR values or differences are the exclusive OR values or difference values of the compact representation substrings of the N first compact representation substrings and at least one corresponding reference descriptor respectively.
- the second compacted representation includes at least one second compacted representation substring.
- the encoding end undergoes compaction processing first to reduce the data volume of the descriptors to be encoded, and then maps the first compact representation to the second compact representation to change the data distribution of the compact representation and further reduce the 3D representation obtained through encapsulation.
- the data volume of the code stream of the map thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the decoding end may perform inverse mapping on the second compact representation obtained by decapsulation to obtain the first compact representation, and then obtain reconstruction data of 3D map points.
- FIG. 17 takes the first compaction to represent the same length of the substring as an example, which is not limited in this embodiment of the present application.
- the first compaction represents that the length of the substring is It can also be different, and the embodiments of the present application do not give examples one by one.
- the at least one first center information is used to indicate the cluster center of at least one sub-feature of the descriptor to be encoded.
- Any sub-feature of the descriptor to be encoded can be understood as a point in a space composed of at least one dimension, and the center here can be the geometric center or centroid or center of gravity of a point or multiple points.
- the center is the point or subfeature itself.
- the plurality of points may include a plurality of sub-features of the descriptor to be encoded, or the plurality of points may include one or more sub-features of the descriptor to be encoded and one or more of the encoded descriptor. multiple sub-features.
- Any piece of first center information may include at least one item such as a number of a center, or a value of at least one dimension of a center.
- the first center information is used to represent a center, and then represent a class of sub-features.
- the same type of sub-features have certain similarities, and different types of sub-features have differences.
- the sum of the distances from each sub-feature to the center in the same type of sub-feature is less than the sum of the distances from each sub-feature to another sub-feature.
- the descriptor to be encoded may be divided into one or more sub-features, and each sub-feature is clustered to obtain one or more first central information.
- the one or more first center information may be numbers of one or more cluster centers.
- the data distribution of the 3D map further reduces the data volume of the code stream of the 3D map obtained through compaction processing and packaging, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- FIG. 19 is a schematic flow chart of a method for decompressing a 3D map provided by an embodiment of the present application.
- the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
- the method includes but is not limited to the following steps:
- reconstructed data of at least one 3D map point is obtained.
- the reconstructed data of the at least one 3D map point can be used for 3D map based positioning.
- At least one second center information of the descriptor to be decoded is obtained by decapsulating the code stream of the 3D map, and according to the at least one second center information of the descriptor to be decoded and the reference corresponding to the descriptor to be decoded At least one first reference center information of the descriptor is obtained to obtain at least one first center information of the descriptor to be decoded, and at least one reconstruction data of a 3D map point is obtained according to the at least one first center information of the descriptor to be decoded.
- the encoding end first undergoes clustering processing, uses the first central information to represent the original descriptor to be encoded, reduces the data volume of the descriptor to be encoded, and then maps at least one first central information to at least one second central information to change
- the data distribution of the central information further reduces the data volume of the code stream of the 3D map obtained through compaction processing and packaging, thereby reducing the resource overhead required for transmitting the code stream of the 3D map.
- the decoding end may perform inverse mapping on the at least one second center information obtained by decapsulation to obtain at least one first center information, and then obtain reconstruction data of 3D map points.
- the 3D map compression method and decompression method of the embodiment of the present application are described in detail above with reference to the accompanying drawings, and the 3D map compression device and decompression device of the embodiment of the present application are introduced below with reference to FIGS. 14 to 16 .
- the 3D map compression device can execute the 3D map compression method of the embodiment of the present application
- the 3D map decompression device can execute the 3D map decompression method of the embodiment of the present application.
- repeated descriptions are appropriately omitted when introducing the apparatus for compressing and decompressing a 3D map according to the embodiment of the present application below.
- the compaction processing module 1401 is configured to perform compaction processing on the descriptor to be coded to obtain a first compact representation of the descriptor to be coded.
- the descriptor to be encoded corresponds to at least one 3D map point in the 3D map.
- the above apparatus may further include: a transmission module 1404 .
- the transmission module 1404 is configured to receive the 3D map request information sent by the electronic device, and send the code stream of the 3D map corresponding to the 3D map request information to the electronic device in response to the 3D map request information.
- the transmission module 1404 is configured to send the code stream of the 3D map to the server.
- the mapping module 1402 is configured to obtain the second compact representation of the descriptor to be coded by looking up the first table according to the first compact representation of the descriptor to be coded and the compact representation of at least one reference descriptor .
- the first table includes at least one encoded compact representation substring and at least one third compact representation substring corresponding to at least one second reference compact representation substring.
- the first compacted representation of the descriptor to be encoded comprises at least one first compacted representation substring of the descriptor to be encoded.
- the at least one encoded compact representation substring comprises at least one first compact representation substring
- the at least one second reference compact representation substring comprises at least one first reference compact representation substring of the at least one reference descriptor.
- the mapping module 1402 is configured to acquire at least A first compacted representation substring corresponds to at least one third compacted representation substring corresponding to at least one first reference compacted representation substring. According to at least one first compacted representation substring and at least one third compacted representation substring corresponding to at least one first reference compacted representation substring, a second compacted representation of the descriptor to be encoded is obtained.
- the mapping module 1402 is configured to: use the XOR value or the difference between at least one compacted representation of the reference descriptor and the first compacted representation of the descriptor to be coded as the second compacted representation of the descriptor to be coded Expressed.
- the 3D map compression apparatus 1400 may implement the 3D map compression method of the embodiment shown in FIG. 5 or FIG. 8 or FIG. 10 or FIG. 12 .
- the 3D map compression apparatus 1400 may implement the 3D map compression method of the embodiment shown in FIG. 5 or FIG. 8 or FIG. 10 or FIG. 12 .
- the embodiment of the present application also provides another device for compressing a 3D map.
- the device for compressing a 3D map may adopt the same structure as that in FIG. 14 .
- the mapping module 1402 is configured to perform clustering processing on the descriptors to be encoded to obtain at least one first center information of the descriptors to be encoded.
- the descriptor to be encoded corresponds to at least one 3D map point in the 3D map.
- the at least one first center information is used to indicate the cluster center of at least one sub-feature of the descriptor to be encoded.
- the mapping module 1402 is further configured to obtain at least one first reference center information of the reference descriptor corresponding to the descriptor to be encoded.
- the reference descriptor corresponds to at least one encoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one second center information of the descriptor to be encoded is obtained according to at least one first center information and at least one first reference center information of the descriptor to be encoded.
- the compaction processing module 1401 is configured to perform compaction processing on at least one second central information of the descriptor to be coded to obtain a compact representation of the descriptor to be coded.
- the encapsulation module 1403 is configured to encapsulate the compact representation to obtain a code stream of the 3D map.
- the foregoing apparatus may further include: a transmission module 1404 .
- the transmission module 1404 is used to receive the 3D map request information sent by the electronic device, and in response to the 3D map request information, send the code stream of the 3D map corresponding to the 3D map request information to the electronic device, or the transmission module 1404 is used to send the code stream of the 3D map to the server The code stream of the 3D map.
- the mapping module 1402 is specifically configured to: according to at least one first center information and at least one first reference center information of the descriptor to be encoded, by looking up the first table to obtain at least A second central message.
- the first table includes at least one coding center information and at least one third center information corresponding to at least one second reference center information.
- the at least one encoding center information includes at least one first center information of the descriptor to be encoded
- the at least one second reference center information includes the at least one first reference center information.
- the mapping module 1402 is specifically configured to: acquire at least one first center information and at least one first reference center information from the first table according to at least one first center information and at least one first reference center information of the descriptor to be encoded Corresponding to at least one third center information.
- the at least one third central information is used as at least one second central information of the descriptor to be encoded.
- the first table is a multidimensional array, and each element in the multidimensional array corresponds to at least one coding center information or at least one second reference center information.
- the mapping module 1402 is specifically configured to: obtain at least one second central information of the descriptor to be encoded by looking up a second table according to at least one first central information of the descriptor to be encoded.
- the second table is established based on the at least one first reference center information.
- the second table includes the at least one first reference center information and at least one fourth center information corresponding to the at least one first center information of the descriptor to be encoded.
- the mapping module 1402 is specifically configured to: acquire at least one fourth central information corresponding to the at least one first central information from the second table according to the at least one first central information of the descriptor to be encoded.
- the at least one fourth central information corresponding to the at least one first central information is used as at least one second central information of the descriptor to be encoded.
- the second table is an at least one-dimensional array, and each element in the at least one-dimensional array corresponds to at least one piece of first central information.
- the mapping module 1402 is specifically configured to: take the XOR value or difference value of at least one first center information of the descriptor to be encoded and at least one first reference center information as the At least one second central information.
- the apparatus for compressing a 3D map may implement the method for compressing a 3D map in the embodiment shown in FIG. 18 .
- the apparatus for compressing a 3D map may implement the method for compressing a 3D map in the embodiment shown in FIG. 18 .
- FIG. 15 is a schematic structural diagram of a device for decompressing a 3D map provided by an embodiment of the present application.
- the apparatus 1500 for decompressing the 3D map may include: a decapsulation module 1501 , an inverse mapping module 1502 and a reconstruction module 1503 .
- the decapsulation module 1501 is configured to decapsulate the code stream of the 3D map to obtain a second compact representation of the descriptor to be decoded.
- the descriptor to be decoded corresponds to at least one 3D map point in the 3D map.
- the demapping module 1502 is configured to obtain a compact representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the at least one reference descriptor corresponds to at least one decoded 3D map point of the 3D map.
- the inverse mapping module 1502 is further configured to obtain the first compacted representation of the descriptor to be decoded according to the second compacted representation of the descriptor to be decoded and the compacted representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the reconstruction module 1503 is configured to obtain reconstruction data of at least one 3D map point according to the first compact representation of the descriptor to be decoded.
- the above apparatus may further include: a transmission module 1504, configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information.
- a transmission module 1504 configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information.
- the demapping module 1502 is configured to: according to the second compact representation of the descriptor to be decoded and the compact representation of at least one reference descriptor corresponding to the descriptor to be decoded, by looking up the first table, obtain the The first compact representation of the descriptor.
- the demapping module 1502 is configured to: obtain the first compact representation of the descriptor to be decoded by looking up the second table according to the second compact representation of the descriptor to be decoded.
- the second table is established based on the compact representation of at least one reference descriptor corresponding to the descriptor to be decoded.
- the inverse mapping module 1502 is configured to: use the XOR value or the sum value of at least one compacted representation of the reference descriptor and the second compacted representation of the descriptor to be decoded as the first A compact representation.
- the 3D map decompression apparatus 1500 may implement the 3D map decompression method in the embodiment shown in FIG. 7 or FIG. 9 or FIG. 11 or FIG. 13 .
- the 3D map decompression apparatus 1500 may implement the 3D map decompression method in the embodiment shown in FIG. 7 or FIG. 9 or FIG. 11 or FIG. 13 .
- the embodiment of the present application also provides another device for decompressing a 3D map.
- the device for decompressing a 3D map may adopt the same structure as that in FIG. 15 .
- the decapsulation module 1501 is configured to decapsulate the code stream of the 3D map to obtain at least one second center information of the descriptor to be decoded.
- the descriptor to be decoded corresponds to at least one 3D map point in the 3D map.
- the demapping module 1502 is configured to obtain at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded.
- the reference descriptor corresponds to at least one decoded 3D map point of the 3D map.
- the at least one first reference center information is used to represent the clustering center of at least one sub-feature of the reference descriptor.
- At least one first center information of the descriptor to be decoded is obtained according to at least one second center information of the descriptor to be decoded and at least one first reference center information of a reference descriptor corresponding to the descriptor to be decoded.
- the reconstruction module 1503 is configured to obtain reconstruction data of the at least one 3D map point according to at least one first center information of the descriptor to be decoded.
- the above apparatus may further include: a transmission module 1504, configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information. Or, it is used to receive the code stream of the 3D map sent by the electronic device.
- a transmission module 1504 configured to send 3D map request information, and receive a code stream of a 3D map corresponding to the 3D map request information. Or, it is used to receive the code stream of the 3D map sent by the electronic device.
- the demapping module 1502 is specifically configured to: according to at least one second central information of the descriptor to be decoded and at least one first reference central information of the reference descriptor corresponding to the descriptor to be decoded, by searching the first A table to obtain at least one first center information of the descriptor to be decoded.
- the demapping module 1502 is specifically configured to: obtain at least one first central information of the descriptor to be decoded by looking up a second table according to at least one second central information of the descriptor to be decoded.
- the second table is established based on at least one first reference center information of the reference descriptor corresponding to the descriptor to be decoded.
- the inverse mapping module 1502 is specifically configured to: the difference between at least one second center information of the descriptor to be decoded and at least one first reference center information of a reference descriptor corresponding to the descriptor to be decoded
- the OR value or the sum value is used as at least one first central information of the descriptor to be decoded.
- the device for decompressing a 3D map may execute the method for decompressing a 3D map in the embodiment shown in FIG. 19 .
- the device for decompressing a 3D map may execute the method for decompressing a 3D map in the embodiment shown in FIG. 19 .
- Fig. 16 is a schematic block diagram of an implementation manner of a decoding device 1600 used in the embodiment of the present application.
- the acquiring device 1600 may include a processor 1601 , a memory 1602 and a bus system 1603 .
- the processor 1601 and the memory 1602 are connected through the bus system 1603, the memory 1602 is used to store instructions, and the processor 1601 is used to execute the instructions stored in the memory 1302 to perform compression or decompression of various 3D maps described in this application. compression method. To avoid repetition, no detailed description is given here.
- the processor 1601 can be a central processing unit (central processing unit, CPU), and the processor 1601 can also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components, etc.
- a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
- the memory 1602 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1602 .
- Memory 1602 may include code and data 16021 accessed by processor 1601 using bus 1603 .
- the memory 1602 may further include an operating system 16023 and an application program 16022, where the application program 16022 includes at least one program that allows the processor 1601 to execute the 3D map compression or decompression method described in this application.
- the application program 16022 may include applications 1 to N, which further include a 3D map compression or decompression application (referred to as a 3D map decoding application) that executes the 3D map compression or decompression method described in this application.
- a 3D map decoding application referred to as a 3D map decoding application
- the bus system 1603 may include not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 1603 in the figure.
- the decoding apparatus 1600 may further include one or more output devices, such as a display 1604 .
- display 1604 may be a touch-sensitive display that incorporates a display with a haptic unit operable to sense touch input.
- the display 1604 can be connected to the processor 1601 via the bus 1603 .
- the decoding device 1600 can implement the 3D map compression method in this application, and can also implement the 3D map decompression method in this application.
- the processor mentioned in the above embodiments may be an integrated circuit chip with signal processing capability.
- each step of the above-mentioned method embodiments may be completed by an integrated logic circuit of hardware in a processor or instructions in the form of software.
- the processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other possible Program logic devices, discrete gate or transistor logic devices, discrete hardware components.
- a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
- the steps of the methods disclosed in the embodiments of the present application may be directly implemented by a hardware coded processor, or executed by a combination of hardware and software modules in the coded processor.
- the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
- the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
- the memories mentioned in the above embodiments may be volatile memories or nonvolatile memories, or may include both volatile and nonvolatile memories.
- the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
- Volatile memory can be random access memory (RAM), which acts as external cache memory.
- RAM random access memory
- SRAM static random access memory
- DRAM dynamic random access memory
- DRAM synchronous dynamic random access memory
- SDRAM double data rate synchronous dynamic random access memory
- ESDRAM enhanced synchronous dynamic random access memory
- SLDRAM direct memory bus random access memory
- direct rambus RAM direct rambus RAM
- the disclosed systems, devices and methods may be implemented in other ways.
- the device embodiments described above are only illustrative.
- the division of the units is only a logical function division. In actual implementation, there may be other division methods.
- multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
- the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
- the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium.
- the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (personal computer, server, or network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disc and other media that can store program codes. .
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
Claims (58)
- 一种3D地图的压缩方法,其特征在于,所述方法包括:对待编码描述子进行紧凑化处理,得到所述待编码描述子的第一紧凑化表示;所述待编码描述子对应所述3D地图中的至少一个3D地图点;获取所述待编码描述子对应的至少一个参考描述子的紧凑化表示;所述至少一个参考描述子对应于所述3D地图的至少一个已编码的3D地图点;根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,得到所述待编码描述子的第二紧凑化表示;对所述第二紧凑化表示进行封装,得到所述3D地图的码流。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:接收电子设备发送的3D地图请求信息,响应于所述3D地图请求信息,向所述电子设备发送所述3D地图请求信息对应的所述3D地图的码流;或者,向服务器发送所述3D地图的码流。
- 根据权利要求1或2所述的方法,其特征在于,所述第一紧凑化表示的数据分布与所述第二紧凑化表示的数据分布不同。
- 根据权利要求3所述的方法,其特征在于,所述待编码描述子的第一紧凑化表示包括所述待编码描述子的至少一个第一紧凑化表示子串,所述待编码描述子的第二紧凑化表示包括所述待编码描述子的至少一个第二紧凑化表示子串;所述至少一个第一紧凑化表示子串与所述至少一个第二紧凑化表示子串不同。
- 根据权利要求4所述的方法,其特征在于,所述至少一个第二紧凑化表示子串中部分或全部小于或等于对应的第一紧凑化表示子串;或者,所述至少一个第二紧凑化表示子串中部分或全部大于或等于对应的第一紧凑化表示子串。
- 根据权利要求1至5任一项所述的方法,其特征在于,所述根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,得到所述待编码描述子的第二紧凑化表示,包括:根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,通过查第一表,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求6所述的方法,其特征在于,所述第一表包括至少一个编码紧凑化表示子串和至少一个第二参考紧凑化表示子串对应的至少一个第三紧凑化表示子串;所述至少一个编码紧凑化表示子串包括所述待编码描述子的至少一个第一紧凑化表示子串,所述至少一个第二参考紧凑化表示子串包括所述至少一个参考描述子的至少一个第一参考紧凑化表示子串;所述根据所述待编码描述子的第一紧凑化表示和所述参考描述子的紧凑化表示,通过查第一表,得到所述待编码描述子的第二紧凑化表示,包括:根据所述待编码描述子的至少一个第一紧凑化表示子串和所述待编码描述子对应的参考描述子的至少一个第一参考紧凑化表示子串,从所述第一表中,获取所述至少一个第一紧凑化表示子串和所述至少一个第一参考紧凑化表示子串对应的至少一个第三紧凑化表示子串;根据所述至少一个第一紧凑化表示子串和所述至少一个第一参考紧凑化表示子串对 应的至少一个第三紧凑化表示子串,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求7所述的方法,其特征在于,所述第一表为多维数组,所述多维数组中的各个元素分别对应至少一个编码紧凑化表示子串或至少一个第二参考紧凑化表示子串。
- 根据权利要求1至5任一项所述的方法,其特征在于,所述根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,得到所述待编码描述子的第二紧凑化表示,包括:根据所述待编码描述子的第一紧凑化表示,通过查第二表,得到所述待编码描述子的第二紧凑化表示;所述第二表是基于所述至少一个参考描述子的紧凑化表示建立的。
- 根据权利要求9所述的方法,其特征在于,所述至少一个参考描述子的紧凑化表示包括至少一个第一参考紧凑化表示子串;所述第二表包括所述至少一个第一参考紧凑化表示子串和所述待编码描述子的至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串;所述根据所述待编码描述子的第一紧凑化表示,通过查第二表,得到所述待编码描述子的第二紧凑化表示,包括:根据所述待编码描述子的至少一个第一紧凑化表示子串,从所述第二表中,获取所述至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串;所述第二表是基于所述至少一个第一参考紧凑化表示子串建立的;根据所述至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求10所述的方法,其特征在于,所述第二表为至少一维数组,所述至少一维数组中的各个元素分别对应至少一个第一紧凑化表示子串。
- 根据权利要求1至5任一项所述的方法,其特征在于,所述根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,得到所述待编码描述子的第二紧凑化表示,包括:将所述至少一个参考描述子的紧凑化表示和所述待编码描述子的第一紧凑化表示的异或值或者差值,作为所述待编码描述子的第二紧凑化表示。
- 根据权利要求1至12任一项所述的方法,其特征在于,所述待编码描述子包括区域描述子或3D地图点描述子。
- 根据权利要求1至13任一项所述的方法,其特征在于,所述待编码描述子对应的至少一个参考描述子,包括:所述待编码描述子之前的已压缩的至少一个描述子;或者,至少一个预设描述子;或者,通过神经网络模型或聚类确定的至少一个描述子。
- 根据权利要求1至14任一项所述的方法,其特征在于,所述紧凑化处理包括量化处理或二值化处理,所述量化处理或二值化处理用于减少所述待编码描述子的比特数。
- 一种3D地图的解压缩方法,其特征在于,所述方法包括:对3D地图的码流进行解封装,得到待解码描述子的第二紧凑化表示;所述待解码描述子对应所述3D地图中的至少一个3D地图点;获取所述待解码描述子对应的至少一个参考描述子的紧凑化表示;所述至少一个参考描述子对应于所述3D地图的至少一个已解码的3D地图点;根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,得到所述待解码描述子的第一紧凑化表示;根据所述待解码描述子的第一紧凑化表示,得到所述至少一个3D地图点的重构数据。
- 根据权利要求16所述的方法,其特征在于,所述方法还包括:发送3D地图请求信息,接收所述3D地图请求信息对应的3D地图的码流。
- 根据权利要求16或17所述的方法,其特征在于,所述第一紧凑化表示的数据分布与所述第二紧凑化表示的数据分布不同。
- 根据权利要求18所述的方法,其特征在于,所述待解码描述子的第一紧凑化表示包括所述待解码描述子的至少一个第一紧凑化表示子串,所述待解码描述子的第二紧凑化表示包括所述待解码描述子的至少一个第二紧凑化表示子串;所述至少一个第一紧凑化表示子串与所述至少一个第二紧凑化表示子串不同。
- 根据权利要求19所述的方法,其特征在于,所述至少一个第二紧凑化表示子串中部分或全部小于或等于对应的第一紧凑化表示子串;或者,所述至少一个第二紧凑化表示子串中部分或全部大于或等于对应的第一紧凑化表示子串。
- 根据权利要求16至20任一项所述的方法,其特征在于,所述根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,得到所述待解码描述子的第一紧凑化表示,包括:根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,通过查第一表,得到所述待解码描述子的第一紧凑化表示。
- 根据权利要求16至20任一项所述的方法,其特征在于,所述根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,得到所述待解码描述子的第一紧凑化表示,包括:根据所述待解码描述子的第二紧凑化表示,通过查第二表,得到所述待解码描述子的第一紧凑化表示;所述第二表是基于所述待解码描述子对应的至少一个参考描述子的紧凑化表示建立的。
- 根据权利要求16至20任一项所述的方法,其特征在于,所述根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,得到所述待解码描述子的第一紧凑化表示,包括:将所述至少一个参考描述子的紧凑化表示和所述待解码描述子的第二紧凑化表示的异或值或者加和值,作为所述待解码描述子的第一紧凑化表示。
- 一种3D地图的压缩装置,其特征在于,所述装置包括:紧凑化处理模块,用于对待编码描述子进行紧凑化处理,得到所述待编码描述子的第一紧凑化表示;所述待编码描述子对应所述3D地图中的至少一个3D地图点;映射模块,用于获取所述待编码描述子对应的至少一个参考描述子的紧凑化表示;所述至少一个参考描述子对应于所述3D地图的至少一个已编码的3D地图点;映射模块,还用于根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,得到所述待编码描述子的第二紧凑化表示;封装模块,用于对所述第二紧凑化表示进行封装,得到所述3D地图的码流。
- 根据权利要求24所述的装置,其特征在于,所述装置还包括:传输模块;所述传输模块,用于接收电子设备发送的3D地图请求信息,响应于所述3D地图请求信息,向所述电子设备发送所述3D地图请求信息对应的所述3D地图的码流;或者,所述传输模块,用于向服务器发送所述3D地图的码流。
- 根据权利要求24或25所述的装置,其特征在于,所述第一紧凑化表示的数据分布与所述第二紧凑化表示的数据分布不同。
- 根据权利要求26所述的装置,其特征在于,所述待编码描述子的第一紧凑化表示包括所述待编码描述子的至少一个第一紧凑化表示子串,所述待编码描述子的第二紧凑化表示包括所述待编码描述子的至少一个第二紧凑化表示子串;所述至少一个第一紧凑化表示子串与所述至少一个第二紧凑化表示子串不同。
- 根据权利要求27所述的装置,其特征在于,所述至少一个第二紧凑化表示子串中部分或全部小于或等于对应的第一紧凑化表示子串;或者,所述至少一个第二紧凑化表示子串中部分或全部大于或等于对应的第一紧凑化表示子串。
- 根据权利要求24至28任一项所述的装置,其特征在于,所述映射模块用于:根据所述待编码描述子的第一紧凑化表示和所述至少一个参考描述子的紧凑化表示,通过查第一表,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求29所述的装置,其特征在于,所述第一表包括至少一个编码紧凑化表示子串和至少一个第二参考紧凑化表示子串对应的至少一个第三紧凑化表示子串;所述至少一个编码紧凑化表示子串包括所述待编码描述子的至少一个第一紧凑化表示子串,所述至少一个第二参考紧凑化表示子串包括所述至少一个参考描述子的至少一个第一参考紧凑化表示子串;所述映射模块用于:根据所述待编码描述子的至少一个第一紧凑化表示子串和所述待编码描述子对应的参考描述子的至少一个第一参考紧凑化表示子串,从所述第一表中,获取所述至少一个第一紧凑化表示子串和所述至少一个第一参考紧凑化表示子串对应的至少一个第三紧凑化表示子串;根据所述至少一个第一紧凑化表示子串和所述至少一个第一参考紧凑化表示子串对应的至少一个第三紧凑化表示子串,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求30所述的装置,其特征在于,所述第一表为多维数组,所述多维数组中的各个元素分别对应至少一个编码紧凑化表示子串或至少一个第二参考紧凑化表示子串。
- 根据权利要求24至28任一项所述的装置,其特征在于,所述映射模块,用于:根据所述待编码描述子的第一紧凑化表示,通过查第二表,得到所述待编码描述子的第二紧凑化表示;所述第二表是基于所述至少一个参考描述子的紧凑化表示建立的。
- 根据权利要求32所述的装置,其特征在于,所述至少一个参考描述子的紧凑化表示包括至少一个第一参考紧凑化表示子串;所述第二表包括所述至少一个第一参考紧凑化表示子串和所述待编码描述子的至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串;所述映射模块,用于:根据所述待编码描述子的至少一个第一紧凑化表示子串,从所 述第二表中,获取所述至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串;所述第二表是基于所述至少一个第一参考紧凑化表示子串建立的;根据所述至少一个第一紧凑化表示子串对应的至少一个第四紧凑化表示子串,得到所述待编码描述子的第二紧凑化表示。
- 根据权利要求33所述的装置,其特征在于,所述第二表为至少一维数组,所述至少一维数组中的各个元素分别对应至少一个第一紧凑化表示子串。
- 根据权利要求24至28任一项所述的装置,其特征在于,所述映射模块,用于:将所述至少一个参考描述子的紧凑化表示和所述待编码描述子的第一紧凑化表示的异或值或者差值,作为所述待编码描述子的第二紧凑化表示。
- 根据权利要求24至35任一项所述的装置,其特征在于,所述待编码描述子包括区域描述子或3D地图点描述子。
- 根据权利要求24至36任一项所述的装置,其特征在于,所述待编码描述子对应的至少一个参考描述子,包括:所述待编码描述子之前的已压缩的至少一个描述子;或者,至少一个预设描述子;或者,通过神经网络模型或聚类确定的至少一个描述子。
- 根据权利要求24至37任一项所述的装置,其特征在于,所述紧凑化处理包括量化处理或二值化处理,所述量化处理或二值化处理用于减少所述待编码描述子的比特数。
- 一种3D地图的解压缩装置,其特征在于,所述装置包括:解封装模块,用于对3D地图的码流进行解封装,得到待解码描述子的第二紧凑化表示;所述待解码描述子对应所述3D地图中的至少一个3D地图点;反映射模块,用于获取所述待解码描述子对应的至少一个参考描述子的紧凑化表示;所述至少一个参考描述子对应于所述3D地图的至少一个已解码的3D地图点;反映射模块,还用于根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,得到所述待解码描述子的第一紧凑化表示;重构模块,用于根据所述待解码描述子的第一紧凑化表示,得到所述至少一个3D地图点的重构数据。
- 根据权利要求39所述的装置,其特征在于,所述装置还包括:传输模块,用于发送3D地图请求信息,接收所述3D地图请求信息对应的3D地图的码流。
- 根据权利要求39或40所述的装置,其特征在于,所述第一紧凑化表示的数据分布与所述第二紧凑化表示的数据分布不同。
- 根据权利要求41所述的装置,其特征在于,所述待解码描述子的第一紧凑化表示包括所述待解码描述子的至少一个第一紧凑化表示子串,所述待解码描述子的第二紧凑化表示包括所述待解码描述子的至少一个第二紧凑化表示子串;所述至少一个第一紧凑化表示子串与所述至少一个第二紧凑化表示子串不同。
- 根据权利要求42所述的装置,其特征在于,所述至少一个第二紧凑化表示子串中部分或全部小于或等于对应的第一紧凑化表示子串;或者,所述至少一个第二紧凑化表示子串中部分或全部大于或等于对应的第一紧凑化表示子串。
- 根据权利要求39至43任一项所述的装置,其特征在于,所述反映射模块,用于:根据所述待解码描述子的第二紧凑化表示和所述待解码描述子对应的至少一个参考描述子的紧凑化表示,通过查第一表,得到所述待解码描述子的第一紧凑化表示。
- 根据权利要求39至43任一项所述的装置,其特征在于,所述反映射模块,用于:根据所述待解码描述子的第二紧凑化表示,通过查第二表,得到所述待解码描述子的第一紧凑化表示;所述第二表是基于所述待解码描述子对应的至少一个参考描述子的紧凑化表示建立的。
- 根据权利要求39至43任一项所述的装置,其特征在于,所述反映射模块,用于:将所述至少一个参考描述子的紧凑化表示和所述待解码描述子的第二紧凑化表示的异或值或者加和值,作为所述待解码描述子的第一紧凑化表示。
- 一种3D地图的压缩方法,其特征在于,所述方法包括:对待编码描述子进行聚类处理,得到所述待编码描述子的至少一个第一中心信息;所述待编码描述子对应所述3D地图中的至少一个3D地图点;所述至少一个第一中心信息用于表示所述待编码描述子的至少一个子特征经所述聚类处理后的中心;获取所述待编码描述子对应的参考描述子的至少一个第一参考中心信息;所述参考描述子对应于所述3D地图的至少一个已编码的3D地图点;所述至少一个第一参考中心信息用于表示所述参考描述子的至少一个子特征经聚类处理后的中心;根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,得到所述待编码描述子的至少一个第二中心信息;对所述待编码描述子的至少一个第二中心信息进行紧凑化处理,得到所述待编码描述子的紧凑化表示;对所述紧凑化表示进行封装,得到3D地图的码流。
- 根据权利要求47所述的方法,其特征在于,所述根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,得到所述待编码描述子的至少一个第二中心信息,包括:根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,通过查第一表,得到所述待编码描述子的至少一个第二中心信息。
- 根据权利要求48所述的方法,其特征在于,所述第一表包括至少一个编码中心信息和至少一个第二参考中心信息对应的至少一个第三中心信息;所述至少一个编码中心信息包括所述至少一个第一中心信息,所述至少一个第二参考中心信息包括所述至少一个第一参考中心信息;所述根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,通过查第一表,得到所述待编码描述子的至少一个第二中心信息,包括:根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,从所述第一表中,获取所述至少一个第一中心信息和所述至少一个第一参考中心信息对应的至少一个第三中心信息;将所述至少一个第三中心信息,作为所述待编码描述子的至少一个第二中心信息。
- 根据权利要求49所述的方法,其特征在于,所述第一表为多维数组,所述多维数组中的各个元素分别对应至少一个编码中心信息或至少一个第二参考中心信息。
- 根据权利要求47所述的方法,其特征在于,所述根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,得到所述待编码描述子的至少一个第二中心信息,包括:根据所述至少一个第一中心信息,通过查第二表,得到所述待编码描述子的至少一个第二中心信息;所述第二表是基于所述至少一个第一参考中心信息建立的。
- 根据权利要求51所述的方法,其特征在于,所述第二表包括所述至少一个第一参考中心信息和所述至少一个第一中心信息对应的至少一个第四中心信息;所述根据所述至少一个第一中心信息,通过查第二表,得到所述待编码描述子的至少一个第二中心信息,包括:根据所述至少一个第一中心信息,从所述第二表中,获取所述至少一个第一中心信息对应的至少一个第四中心信息;将所述至少一个第一中心信息对应的至少一个第四中心信息,作为所述待编码描述子的至少一个第二中心信息。
- 根据权利要求52所述的方法,其特征在于,所述第二表为至少一维数组,所述至少一维数组中的各个元素分别对应至少一个第一中心信息。
- 根据权利要求47所述的方法,其特征在于,所述根据所述至少一个第一中心信息和所述至少一个第一参考中心信息,得到所述待编码描述子的至少一个第二中心信息,包括:将所述至少一个第一中心信息和所述至少一个第一参考中心信息的异或值或者差值,作为所述待编码描述子的至少一个第二中心信息。
- 一种3D地图的解压缩方法,其特征在于,所述方法包括:对3D地图的码流进行解封装,得到待解码描述子的至少一个第二中心信息;所述待解码描述子对应所述3D地图中的至少一个3D地图点;获取所述待解码描述子对应的参考描述子的至少一个第一参考中心信息;所述参考描述子对应于所述3D地图的至少一个已解码的3D地图点;所述至少一个第一参考中心信息用于表示所述参考描述子的至少一个子特征经聚类处理后的中心;根据所述至少一个第二中心信息和所述至少一个第一参考中心信息,得到所述待解码描述子的至少一个第一中心信息;根据所述待解码描述子的至少一个第一中心信息,得到所述至少一个3D地图点的重构数据。
- 一种3D地图的压缩装置,其特征在于,包括:一个或多个处理器;存储器,用于存储一个或多个程序;当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-15中任一项所述的方法或如权利要求47-54中任一项所述的方法。
- 一种3D地图的解压缩装置,其特征在于,包括:一个或多个处理器;存储器,用于存储一个或多个程序;当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求16-23中任一项所述的方法或如权利要求55所述的方法。
- 一种计算机可读存储介质,其特征在于,包括计算机程序,所述计算机程序在计算机上被执行时,使得所述计算机执行权利要求1-23中任一项所述的方法或权利要求47-55任一项所述的方法。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP22815272.4A EP4336779A1 (en) | 2021-06-04 | 2022-05-31 | 3d map compression method and apparatus, and 3d map decompression method and apparatus |
KR1020247000350A KR20240017917A (ko) | 2021-06-04 | 2022-05-31 | 3d 맵 압축 방법 및 장치, 및 3d 맵 압축해제 방법 및 장치 |
JP2023574721A JP2024522154A (ja) | 2021-06-04 | 2022-05-31 | 3次元マップ圧縮方法および装置、並びに、3次元マップ復元方法および装置 |
US18/528,022 US20240104781A1 (en) | 2021-06-04 | 2023-12-04 | 3D Map Compression Method and Apparatus, and 3D Map Decompression Method and Apparatus |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110627636.9 | 2021-06-04 | ||
CN202110627636 | 2021-06-04 | ||
CN202110823842.7 | 2021-07-21 | ||
CN202110823842.7A CN115442338A (zh) | 2021-06-04 | 2021-07-21 | 3d地图的压缩、解压缩方法和装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/528,022 Continuation US20240104781A1 (en) | 2021-06-04 | 2023-12-04 | 3D Map Compression Method and Apparatus, and 3D Map Decompression Method and Apparatus |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022253228A1 true WO2022253228A1 (zh) | 2022-12-08 |
Family
ID=84272004
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/096261 WO2022253228A1 (zh) | 2021-06-04 | 2022-05-31 | 3d地图的压缩、解压缩方法和装置 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20240104781A1 (zh) |
EP (1) | EP4336779A1 (zh) |
JP (1) | JP2024522154A (zh) |
KR (1) | KR20240017917A (zh) |
CN (1) | CN115442338A (zh) |
WO (1) | WO2022253228A1 (zh) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103218427A (zh) * | 2013-04-08 | 2013-07-24 | 北京大学 | 局部描述子的提取方法、图像检索方法及图像匹配方法 |
CN110866953A (zh) * | 2019-10-31 | 2020-03-06 | Oppo广东移动通信有限公司 | 地图构建方法及装置、定位方法及装置 |
CN111639147A (zh) * | 2020-04-24 | 2020-09-08 | 视辰信息科技(上海)有限公司 | 一种地图压缩方法、系统及计算机可读存储介质 |
CN111937402A (zh) * | 2018-04-11 | 2020-11-13 | 索尼公司 | 图像处理装置和方法 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102968632A (zh) * | 2012-10-15 | 2013-03-13 | 北京大学 | 获取图像的紧凑全局特征描述子的方法及图像检索方法 |
CN103955690B (zh) * | 2014-04-15 | 2017-03-01 | 合肥工业大学 | 一种紧凑的图像局部特征描述子的构造方法 |
CN105868814A (zh) * | 2015-01-23 | 2016-08-17 | 阿里巴巴集团控股有限公司 | 图像码的解析方法及装置、图像码的生成方法及装置 |
US9799130B1 (en) * | 2015-10-27 | 2017-10-24 | Google Inc. | Lossless spatial feature descriptor compression |
FR3068556A1 (fr) * | 2017-06-29 | 2019-01-04 | B<>Com | Procede de decodage d'une image, procede de codage, dispositifs, equipement terminal et programmes d'ordinateurs associes |
-
2021
- 2021-07-21 CN CN202110823842.7A patent/CN115442338A/zh active Pending
-
2022
- 2022-05-31 KR KR1020247000350A patent/KR20240017917A/ko active Search and Examination
- 2022-05-31 WO PCT/CN2022/096261 patent/WO2022253228A1/zh active Application Filing
- 2022-05-31 JP JP2023574721A patent/JP2024522154A/ja active Pending
- 2022-05-31 EP EP22815272.4A patent/EP4336779A1/en active Pending
-
2023
- 2023-12-04 US US18/528,022 patent/US20240104781A1/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103218427A (zh) * | 2013-04-08 | 2013-07-24 | 北京大学 | 局部描述子的提取方法、图像检索方法及图像匹配方法 |
CN111937402A (zh) * | 2018-04-11 | 2020-11-13 | 索尼公司 | 图像处理装置和方法 |
CN110866953A (zh) * | 2019-10-31 | 2020-03-06 | Oppo广东移动通信有限公司 | 地图构建方法及装置、定位方法及装置 |
CN111639147A (zh) * | 2020-04-24 | 2020-09-08 | 视辰信息科技(上海)有限公司 | 一种地图压缩方法、系统及计算机可读存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN115442338A (zh) | 2022-12-06 |
EP4336779A1 (en) | 2024-03-13 |
JP2024522154A (ja) | 2024-06-11 |
US20240104781A1 (en) | 2024-03-28 |
KR20240017917A (ko) | 2024-02-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112381828B (zh) | 基于语义和深度信息的定位方法、装置、介质与设备 | |
WO2021088497A1 (zh) | 虚拟物体显示方法、全局地图更新方法以及设备 | |
WO2023051383A1 (zh) | 一种设备定位方法、设备及系统 | |
WO2022253228A1 (zh) | 3d地图的压缩、解压缩方法和装置 | |
WO2022252345A1 (zh) | 3d地图的压缩、解压缩方法和装置 | |
WO2022252236A1 (zh) | 3d地图的编解码方法及装置 | |
WO2022252238A1 (zh) | 3d地图的压缩、解压缩方法和装置 | |
WO2022252237A1 (zh) | 3d地图的编解码方法及装置 | |
WO2022252234A1 (zh) | 3d地图的编码装置和方法 | |
US20240119639A1 (en) | Apparatus and method for decoding 3d map, and encoded bitstream of 3d map | |
JP2024523186A (ja) | 3dマップを索出する方法、装置、及びコンピュータプログラム | |
JP2024523816A (ja) | 3dマップを圧縮及び圧縮解除するための方法及び装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22815272 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2023574721 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022815272 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2022815272 Country of ref document: EP Effective date: 20231207 |
|
ENP | Entry into the national phase |
Ref document number: 20247000350 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020247000350 Country of ref document: KR |
|
NENP | Non-entry into the national phase |
Ref country code: DE |