WO2022252346A1 - 3d地图的检索方法和装置 - Google Patents

3d地图的检索方法和装置 Download PDF

Info

Publication number
WO2022252346A1
WO2022252346A1 PCT/CN2021/105624 CN2021105624W WO2022252346A1 WO 2022252346 A1 WO2022252346 A1 WO 2022252346A1 CN 2021105624 W CN2021105624 W CN 2021105624W WO 2022252346 A1 WO2022252346 A1 WO 2022252346A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
data
descriptors
retrieval
descriptor
Prior art date
Application number
PCT/CN2021/105624
Other languages
English (en)
French (fr)
Inventor
曹潇然
蔡康颖
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PCT/CN2021/098482 external-priority patent/WO2022252234A1/zh
Priority claimed from PCT/CN2021/098483 external-priority patent/WO2022252235A1/zh
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202180099018.XA priority Critical patent/CN117460972A/zh
Priority to EP21943703.5A priority patent/EP4336223A1/en
Priority to KR1020247000191A priority patent/KR20240017906A/ko
Publication of WO2022252346A1 publication Critical patent/WO2022252346A1/zh
Priority to US18/526,846 priority patent/US20240111793A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
    • G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/42—Determining position
    • G01S19/45—Determining position by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/487—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/189—Recording image signals; Reproducing recorded image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present application relates to positioning technology, in particular to a method and device for retrieving 3D maps.
  • Virtual reality Virtual Reality
  • AR Augmented Reality
  • MR Mixed reality
  • This type of technology can create virtual reality and superimpose virtual reality and real world to generate new visual environment and interactive experience.
  • electronic devices need to determine their own pose information in the current environment, so as to accurately realize the fusion of virtual objects and real scenes.
  • a typical solution is: the electronic device receives a three-dimensional (3-dimension, 3D) map of its environment from a server or other devices, and then passes The local sensor collects visual information in the environment, and searches the downloaded 3D map according to the collected visual information to determine the current pose of the electronic device.
  • 3D three-dimensional
  • the original 3D map usually contains a huge amount of data, and the amount of calculation for retrieval is usually huge, which consumes a lot of computing resources and takes a long time, which affects the user experience.
  • Embodiments of the present application provide a 3D map retrieval method and device, which can improve retrieval performance.
  • the embodiment of the present application provides a method for retrieving a 3D map
  • the method may include: extracting the binary data of S 3D map descriptors from the compressed data of S 3D map descriptors, and the S 3D map
  • the descriptor corresponds to multiple 3D map points in the 3D map.
  • the i-th level retrieval is performed in the binarized data of the S 3D map descriptors to obtain P 3D map descriptors.
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device.
  • the decompression process (full decompression process) of the compressed data of the P 3D map descriptors includes at least the mth level of decompression.
  • the i-th level search is performed in the binarized data of the S 3D map descriptors, and P 3D map descriptors are obtained by screening.
  • P 3D map The compressed data of the descriptor is decompressed at the mth level to obtain the reconstructed data of the P 3D map descriptors.
  • the jth step is performed in the reconstructed data of the P 3D map descriptors Level retrieval to filter out fewer 3D map descriptors.
  • the first-level retrieval of at least two-level retrieval uses binary data in the compressed data, which can increase the retrieval speed
  • the other-level retrieval uses the reconstructed data obtained by decompression, which can ensure retrieval accuracy.
  • the retrieval method for the 3D map provided in the embodiment of the present application can improve the retrieval performance.
  • the 3D map retrieval method provided by the embodiment of the present application can improve retrieval performance.
  • the above-mentioned m-th level decompression may be staged decompression, which may be understood as a sub-process of complete decompression, or may be understood as partial decompression.
  • staged decompression may be understood as a sub-process of complete decompression, or may be understood as partial decompression.
  • the above-mentioned mth stage decompression may be full decompression.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are matched with the 3D map points corresponding to the retrieval descriptors, and N is a positive integer, 0 ⁇ N ⁇ Q, and the above method may further include: performing positioning according to the 3D map points corresponding to the N 3D map descriptors, so as to obtain the pose information of the electronic device.
  • positioning can be performed according to the 3D map points corresponding to the N 3D map descriptors, which is beneficial to improve positioning performance.
  • the retrieval method adopted by the i-level retrieval is a retrieval method based on the first distance
  • the retrieval method adopted by the j-level retrieval is a retrieval method based on the second distance
  • the first distance may include a distance obtained by using binarized data, such as a Hamming distance.
  • the first distance may be the Hamming distance.
  • the first distance may be a distance obtained by taking the absolute value of the difference between corresponding bits of the two binarized data and adding all the absolute values of the difference.
  • the retrieval based on the first distance refers to calculating the first distance to determine the degree of correlation or similarity to filter the 3D map descriptors.
  • the second distance may include, but is not limited to, Euclidean distance, inner product distance, cosine distance, Manhattan distance, and the like.
  • the retrieval based on the second distance refers to calculating the second distance to determine the degree of correlation or similarity to filter the 3D map descriptors.
  • performing retrieval based on the first distance can increase retrieval speed, and performing retrieval based on the second distance can improve retrieval accuracy.
  • the method may further include: receiving the retrieval descriptor, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor.
  • the execution subject (which may be an electronic device or a server (such as a cloud server)) for executing the retrieval method described in the first aspect may receive retrieval descriptors sent by other devices.
  • the execution subject for executing the retrieval method described in the first aspect receives retrieval descriptors collected and extracted by other electronic devices. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the method may further include: receiving the visual information, extracting a retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binary data of the retrieval descriptor;
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the first aspect may receive visual information sent by other devices, and extract retrieval descriptors therefrom. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the method may further include: responding to the operation of collecting visual information input by the user, triggering the sensor to collect visual information from the real environment, obtaining the visual information, and obtaining the visual information from the visual information
  • the retrieval descriptor is extracted, and the retrieval descriptor is binarized to obtain binarized data of the retrieval descriptor.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the first aspect may collect the visual information and extract the retrieval descriptor from it. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the 3D map retrieval method in the embodiment of the present application can retrieve retrieval descriptors in different application scenarios in the 3D map, and has wide applicability.
  • the method further includes: decompressing the compressed data of the Q 3D map descriptors at the kth level to obtain the reconstructed data of the Q 3D map descriptors.
  • the r-th level retrieval is performed in the reconstructed data of the Q 3D map descriptors to obtain N 3D map descriptors.
  • the decompression process of the compressed data of P 3D map descriptors includes m-level decompression (stage decompression) and k-th stage decompression (stage decompression), r and k are Positive integer, m ⁇ k, j ⁇ r ⁇ L.
  • the i-th level search is performed in the binarized data of the S 3D map descriptors, and P 3D map descriptors are obtained by screening.
  • P 3D map descriptors The compressed data is decompressed at the mth level (staged decompression) to obtain the reconstructed data of P 3D map descriptors.
  • An r-th level search is performed on the reconstructed data of the map descriptor to obtain N 3D map descriptors, which can be used for positioning.
  • At least one of the three levels of retrieval uses binarized data in compressed data, which can improve retrieval speed, and the other two levels of retrieval use reconstructed data obtained by different decompression, which can improve retrieval speed while ensuring retrieval accuracy .
  • the degree of decompression or the degree of distortion of the reconstructed data obtained by the m-level decompression and the k-th level of decompression are different.
  • the compressed data of the P 3D map descriptors includes the binarized data and quantized data of the P 3D map descriptors, and the m-th decompression is performed on the compressed data of the P 3D map descriptors to obtain
  • the reconstructed data of the P 3D map descriptors includes: dequantizing the respective quantized data of the P 3D map descriptors to obtain P dequantized data, and the P dequantized data are used as the P 3D map descriptors Restructure data.
  • Decompressing the compressed data of the Q 3D map descriptors at the kth level to obtain the reconstructed data of the Q 3D map descriptors including: dequantizing the respective quantized data of the Q 3D map descriptors to obtain Q
  • the reconstructed data of the Q 3D map descriptors are obtained according to the Q dequantized data and the binarized data of the Q 3D map descriptors.
  • the respective dequantized data of the P 3D map descriptors are obtained, based on the respective dequantized data of the P 3D map descriptors (used in this Reconstructed data retrieved at the first level) to perform j-level retrieval, and obtain Q 3D map descriptors by screening, and dequantize the respective quantized data of the Q 3D map descriptors to obtain Q dequantized data, according to the Q
  • the dequantization data and the binarized data of each of the Q 3D map descriptors are obtained to obtain the respective reconstruction data of the Q 3D map descriptors, and the r-level retrieval is performed based on the respective reconstruction data of the Q 3D map descriptors to obtain N 3D map descriptors.
  • the distortion degree of the reconstructed data retrieved at the j-level is greater than that of the reconstructed data retrieved at the r-level, that is, the rough retrieval is performed first to improve the retrieval speed, and then the fine retrieval is performed to improve the retrieval accuracy.
  • the compressed data of the P 3D map descriptors includes the respective binarized data and quantized data of the P 3D map descriptors, and the compression of the P 3D map descriptors
  • the data is decompressed at the mth level to obtain the reconstructed data of P 3D map descriptors, including: performing dequantization processing on the respective quantized data of the P 3D map descriptors to obtain P dequantized data, and according to the P dequantized data data and the respective binarized data of the P 3D map descriptors to obtain the respective reconstruction data of the P 3D map descriptors.
  • the jth level retrieval may be the last level of the multi-level retrieval, and the last level is retrieved based on the reconstructed data obtained through complete decompression, which can improve the accuracy of the retrieval.
  • the S 3D map descriptors are S representative 3D map descriptors
  • the S representative 3D map descriptors respectively correspond to at least one data set
  • each data set of at least one data set includes at least one 3D map Descriptor, according to the binary data of the retrieval descriptor, perform the i-th level retrieval in the binary data of the S 3D map descriptors to obtain P 3D map descriptors, including: according to the binary data of the retrieval descriptor
  • the i-th level search is performed in the binarized data representing S 3D map descriptors to obtain at least one representative 3D map descriptor, and at least one representative 3D map descriptor is used to represent the 3D map in the data set corresponding to each Descriptor, as P 3D map descriptors.
  • the 3D map descriptors in each data set have certain correlation or similarity.
  • the representative 3D map descriptor is used to characterize common or similar features of the 3D map descriptors in the corresponding data set.
  • P may be greater than S. P ⁇ T, S ⁇ T.
  • one-level retrieval is performed on a small number of representative 3D map descriptors to improve the retrieval speed, and then one-level or multi-level retrieval can be performed in the data set corresponding to the representative 3D map descriptor to improve retrieval accuracy Rate.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • Q ⁇ P ⁇ S ⁇ T That is, the number of 3D map descriptors in multi-level retrieval decreases step by step.
  • the method may further include: according to each component of the retrieval descriptor and the corresponding component of the preset threshold vector, determining the size relationship between each component of the retrieval descriptor and the corresponding component of the preset threshold vector. Binarize the size relationship to obtain the binarized data of the retrieval descriptor.
  • each component of the retrieval descriptor and the corresponding component of the preset threshold vector determine the size relationship between each component of the retrieval descriptor and the corresponding component of the preset threshold vector, including: Each component of the sub is subtracted from the corresponding component of the preset threshold vector to obtain the difference of each component; according to the difference of each component, the sign of each component is determined.
  • Performing binary processing on the size relationship to obtain binary data of the retrieval descriptor includes: performing binary processing on each component symbol to obtain binary data of the retrieval descriptor.
  • the embodiment of the present application provides a method for retrieving a 3D map.
  • the method may include: extracting the first binarized data of the S 3D map descriptors from the compressed data of the S 3D map descriptors, and the S The 3D map descriptor corresponds to multiple 3D map points in the 3D map.
  • the i-th level search is performed on the first binarized data of the S 3D map descriptors to obtain P 3D map descriptors.
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device.
  • the j-th level retrieval is performed in the second binarization data of P 3D map descriptors to obtain Q 3D map descriptors
  • S, P, Q, i and j is a positive integer
  • T represents the total number of 3D map descriptors in the 3D map
  • j i+1, 1 ⁇ i ⁇ L, 1 ⁇ j ⁇ L
  • L represents the total number of retrieval levels of the 3D map or a threshold value of the retrieval level
  • L is a positive integer greater than 1.
  • the first binarized data of S 3D map descriptors are extracted from the compressed data of S 3D map descriptors, and according to the first binarized data of the retrieved descriptors, the S 3D map
  • the i-level search is performed on the first binarized data of the descriptor, and P 3D map descriptors are obtained by screening. Extract the second binarized data of P 3D map descriptors from the compressed data of P 3D map descriptors, according to the second binarized data of the retrieved descriptors, in the second binary data of P 3D map descriptors
  • the j-level retrieval is performed in the 2D data to filter out a smaller number of 3D map descriptors.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • the first binarized data and the second binarized data may be binarized data used in any two levels in the multi-level search, and their binarized processing methods may be different, or their lengths may be different.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors match the 3D map points corresponding to the retrieval descriptors, and N is positive Integer, 0 ⁇ N ⁇ Q, the above method further includes: performing positioning according to the 3D map points corresponding to the N 3D map descriptors, so as to obtain the pose information of the electronic device.
  • the i-th level retrieval and the j-th level retrieval adopt the retrieval method based on the first distance
  • the P 3D map descriptors belong to the S 3D map descriptors
  • the P 3D map The position of the first binarization data of each 3D map descriptor of the descriptor in the compressed data of the 3D map descriptor, which is different from the position of the second binarization data of the 3D map descriptor in the description of the 3D map position in the compressed data of the child, where P ⁇ S.
  • this position may be a start position, or an end position, etc.
  • the length of the first binarized data of each 3D map descriptor of the P 3D map descriptors is smaller than the length of the second binary data of each 3D map descriptor of the P 3D map descriptors.
  • the length of the valued data is smaller than the length of the first binarized data of each 3D map descriptor of the P 3D map descriptors.
  • the length of the first binarized data of each 3D map descriptor in the i-level retrieval is relatively short, which can improve the retrieval speed
  • the length of the second binarized data of each 3D map descriptor in the j-level retrieval Longer can improve retrieval accuracy.
  • the above method further includes: receiving the retrieval descriptor, and performing binarization processing on the retrieval descriptor to obtain the first binarized data and the second binary data of the retrieval descriptor. Value data.
  • the execution subject (which may be an electronic device or a server (such as a cloud server)) for executing the retrieval method described in the second aspect may receive retrieval descriptors sent by other devices.
  • the execution subject for executing the retrieval method described in the second aspect receives retrieval descriptors collected and extracted by other electronic devices. Then perform binarization processing on the retrieval descriptor to obtain first binarized data and second binarized data of the retrieval descriptor.
  • the above method further includes: receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain the retrieval description The first binarized data and the second binarized data of the child.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the second aspect may receive visual information sent by other devices, and extract retrieval descriptors therefrom. Then perform binarization processing on the retrieval descriptor to obtain first binarized data and second binarized data of the retrieval descriptor.
  • the above method further includes: responding to the operation of collecting visual information input by the user, triggering the sensor to collect visual information from the real environment, obtaining the visual information, and extracting from the visual information
  • the search descriptor and performing binarization processing on the search descriptor to obtain first binarized data and second binarized data of the search descriptor.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the second aspect may collect the visual information and extract retrieval descriptors therefrom. Then perform binarization processing on the retrieval descriptor to obtain first binarized data and second binarized data of the retrieval descriptor.
  • the length of the first binarized data of the retrieval descriptor is equal to the length of the first binarized data of each of the S 3D map descriptors, and/or, The length of the second binarized data of the retrieval descriptor is equal to the length of the second binarized data of each 3D map descriptor of the S 3D map descriptors.
  • the search speed can be improved.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the set includes at least one 3D map descriptor, and according to the first binarized data of the search descriptor, the ith-level search is performed on the first binarized data of the S 3D map descriptors to obtain P
  • the 3D map descriptor including: according to the first binarized data of the search descriptor, perform i-level retrieval in the first binarized data of the S representative 3D map descriptors, so as to obtain at least one representative 3D map descriptors: the at least one representative 3D map descriptor in the data set corresponding to each of the 3D map descriptors is used as the P 3D map descriptors.
  • Q ⁇ P ⁇ S ⁇ T That is, the number of 3D map descriptors in multi-level retrieval decreases step by step.
  • the embodiment of the present application provides a method for retrieving a 3D map.
  • the method may include: decompressing the compressed data of the S 3D map descriptors at the mth level to obtain the first Reconstructing data, the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • an i-th level retrieval is performed on the first reconstructed data of the S 3D map descriptors to obtain P 3D map descriptors.
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device.
  • the decompression process of the compressed data of the S 3D map descriptors includes the k-level decompression, and the k-level decompression Includes m-th stage decompression.
  • the compressed data of the S 3D map descriptors are decompressed at the mth stage to obtain the first reconstructed data of the S 3D map descriptors.
  • An i-level search is performed on the first reconstructed data of the S 3D map descriptors, and P 3D map descriptors are screened out.
  • Decompress the compressed data of the P 3D map descriptors at the kth level to obtain the second reconstructed data of the P 3D map descriptors.
  • the j-th level retrieval is performed in the second reconstructed data to filter out a smaller number of 3D map descriptors.
  • one level of retrieval is staged decompression, which can improve the retrieval speed, and the decompression differential settings of at least two levels of retrieval can ensure retrieval accuracy.
  • the degree of decompression or the degree of distortion of the reconstructed data of the 3D map descriptor used by any two levels of retrieval are different. Compared with retrieving in the reconstructed data of the fully decompressed 3D map, the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors match the 3D map points corresponding to the retrieval descriptors , N is a positive integer, 0 ⁇ N ⁇ Q, the above method further includes: performing positioning according to the 3D map points corresponding to the N 3D map descriptors, so as to obtain the pose information of the electronic device.
  • positioning can be performed according to the 3D map points corresponding to the N 3D map descriptors, which is beneficial to improve positioning performance.
  • the i-th level retrieval and the j-th level retrieval adopt a retrieval method based on the second distance
  • the P 3D map descriptors belong to the S 3D map Descriptor
  • the distortion degree of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors, and the distortion of the second reconstructed data of each 3D map descriptor of the P 3D map descriptors degree is different
  • the distortion degree of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors is used to represent the first reconstructed data of each 3D map descriptor and the corresponding original 3D map descriptor
  • the degree of difference between the second reconstruction data of each 3D map descriptor of the P 3D map descriptors is used to represent the second reconstruction data of each 3D map descriptor and the corresponding original 3D
  • the distortion degree of the first reconstruction data of each 3D map descriptor of the P 3D map descriptors is greater than the second reconstruction data of each 3D map descriptor of the P 3D map descriptors Distortion of structured data.
  • the degree of distortion of the first reconstructed data of each 3D map descriptor in the i-th level retrieval may be greater than the degree of distortion of the second reconstructed data of each 3D map descriptor among the P 3D map descriptors.
  • the distortion degree of the previous retrieval is greater than that of the subsequent retrieval, so as to perform fast retrieval in the previous retrieval and improve the retrieval speed, and perform fine retrieval in the latter retrieval to improve retrieval accuracy Rate.
  • the above method further includes: receiving the retrieval descriptor, and acquiring part or all of the data of the retrieval descriptor.
  • the execution subject (which may be an electronic device or a server (such as a cloud server)) for executing the retrieval method described in the third aspect may receive retrieval descriptors sent by other devices.
  • the execution subject for executing the retrieval method described in the third aspect receives retrieval descriptors collected and extracted by other electronic devices. Afterwards, part or all of the data of the retrieval descriptor is obtained according to retrieval requirements.
  • the above method further includes: receiving the visual information, extracting the retrieval descriptor from the visual information, and acquiring part or all of the data of the retrieval descriptor.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the third aspect may receive visual information sent by other devices, and extract retrieval descriptors therefrom. Afterwards, part or all of the data of the retrieval descriptor is obtained according to retrieval requirements.
  • the above method further includes: responding to the operation of collecting visual information input by the user, triggering the sensor to collect visual information from the real environment, obtaining the visual information, and extracting from the visual information
  • the search descriptor and obtain part or all of the data of the search descriptor.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the third aspect may collect the visual information and extract the retrieval descriptor from it. Afterwards, part or all of the data of the retrieval descriptor is obtained according to retrieval requirements.
  • the 3D map retrieval method in the embodiment of the present application can retrieve retrieval descriptors in different application scenarios in the 3D map, and has wide applicability.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the collection includes at least one 3D map descriptor, and according to the part or all data of the retrieval descriptor, the i-th level retrieval is performed in the first reconstructed data of the S 3D map descriptors to obtain P 3D maps Descriptors, including: according to part or all of the data of the retrieval descriptors, perform the i-th level retrieval in the first reconstructed data representing the S representative 3D map descriptors, so as to obtain at least one representative 3D map descriptors ; Using the at least one representative 3D map descriptor in the data set corresponding to each of the 3D map descriptors as the P 3D map descriptors.
  • the 3D map descriptors in each data set have certain correlation or similarity.
  • the representative 3D map descriptor is used to characterize common or similar features of the 3D map descriptors in the corresponding data set.
  • P may be greater than S. P ⁇ T, S ⁇ T.
  • one-level retrieval is performed on a small number of representative 3D map descriptors to improve the retrieval speed, and then one-level or multi-level retrieval can be performed in the data set corresponding to the representative 3D map descriptor to improve retrieval accuracy Rate.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • Q ⁇ P ⁇ S ⁇ T That is, the number of 3D map descriptors in multi-level retrieval decreases step by step.
  • the embodiment of the present application provides a method for retrieving a 3D map
  • the method may include: decompressing the compressed data of S 3D map descriptors at the mth level to obtain reconstructed data of S 3D map descriptors , the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the i-th level retrieval is performed in the reconstructed data of the S 3D map descriptors to obtain P 3D map descriptors; the retrieval descriptor is obtained from the electronic device
  • the features corresponding to the real environment are extracted from the visual information collected by the sensor.
  • At least one level of retrieval is decompression in at least two levels of retrieval, which can improve retrieval accuracy
  • at least one level of retrieval in at least two levels of retrieval is extraction from compressed data, which can improve retrieval speed.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors.
  • Point matching N is a positive integer, 0 ⁇ N ⁇ Q
  • the method further includes: performing positioning according to the 3D map points corresponding to the N 3D map descriptors, so as to obtain the pose information of the electronic device.
  • the retrieval method adopted by the i-th level retrieval is a retrieval method based on the second distance
  • the retrieval method adopted by the j-th level retrieval is a retrieval method based on the first distance
  • the retrieval method based on the second distance can improve the retrieval accuracy
  • the retrieval method based on the first distance can increase the retrieval speed, thereby achieving a comprehensive improvement of retrieval accuracy and retrieval speed.
  • the above method further includes: receiving the retrieval descriptor, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor;
  • the execution subject of the retrieval method described in the aspect may receive retrieval descriptors sent by other devices.
  • the execution subject for executing the retrieval method described in the first aspect receives retrieval descriptors collected and extracted by other electronic devices. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the above method further includes: receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain the retrieval description binarized data;
  • the executive body (which can be an electronic device or a server) used to execute the retrieval method described in the fourth aspect can receive visual information sent by other devices, and extract retrieval descriptors therefrom. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the above method further includes: responding to the operation of collecting visual information input by the user, triggering the sensor to collect visual information from the real environment, obtaining the visual information, and extracting from the visual information
  • the retrieval descriptor and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor.
  • the execution subject (which may be an electronic device or a server) for executing the retrieval method described in the fourth aspect may receive visual information sent by other devices, and extract retrieval descriptors therefrom. Afterwards, the retrieval descriptor is binarized to obtain the binarized data of the retrieval descriptor.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the collection includes at least one 3D map descriptor, and according to the part or all data of the search descriptor, the i-th level retrieval is performed in the reconstructed data of the S 3D map descriptors to obtain P 3D map descriptors , comprising: according to part or all of the data of the search descriptor, performing an i-th level search in the reconstructed data representing the S representative 3D map descriptors to obtain at least one representative 3D map descriptor; At least one representative 3D map descriptor in the data set corresponding to each of the 3D map descriptors is used as the P 3D map descriptors.
  • the 3D map descriptors in each data set have certain correlation or similarity.
  • the representative 3D map descriptor is used to characterize common or similar features of the 3D map descriptors in the corresponding data set.
  • P may be greater than S. P ⁇ T, S ⁇ T.
  • one-level retrieval is performed on a small number of representative 3D map descriptors to improve the retrieval speed, and then one-level or multi-level retrieval can be performed in the data set corresponding to the representative 3D map descriptor to improve retrieval accuracy Rate.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • Q ⁇ P ⁇ S ⁇ T That is, the number of 3D map descriptors in multi-level retrieval decreases step by step.
  • the retrieval descriptor includes an area descriptor or a 3D map point descriptor.
  • the S 3D map descriptors may include S area descriptors or S 3D map point descriptors.
  • the embodiment of the present application provides a device for retrieving a 3D map.
  • the device may be a chip or a system-on-a-chip in an electronic device or a server, and may also be used in an electronic device or a server to implement the first aspect or the first aspect.
  • the retrieval device of the 3D map includes: a retrieval module, an extraction module and a decompression module.
  • the extraction module is used to extract the binarized data of S 3D map descriptors from the compressed data of S 3D map descriptors, and the S 3D map descriptors correspond to multiple A 3D map point; a retrieval module, configured to perform an i-th level retrieval in the binarized data of the S 3D map descriptors according to the binarized data of the retrieval descriptors, to obtain P 3D map descriptors;
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device;
  • the decompression module is used to perform m-level decompression on the compressed data of the P 3D map descriptors, Obtaining the reconstruction data of the P 3D map descriptors; the decompression process of the compressed data of the P 3D map descriptors at least includes the decompression at the mth level;
  • the retrieval module is configured to Some or all of the data of the descriptor is retrieved at the
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors.
  • Point matching N is a positive integer, 0 ⁇ N ⁇ Q
  • the device also includes: a pose determination module; the pose determination module performs positioning according to the 3D map points corresponding to the N 3D map descriptors, to The pose information of the electronic device is obtained.
  • the retrieval method adopted by the i-th level retrieval is a retrieval method based on the first distance
  • the retrieval method adopted by the j-th level retrieval is a retrieval method based on the second distance.
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a binarization of the retrieval descriptor data; or, receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor; or , in response to the operation of collecting visual information input by the user, triggering the sensor to collect visual information on the real environment to obtain the visual information, extracting the retrieval descriptor from the visual information, and describing the retrieval perform binarization processing to obtain the binarized data of the retrieval descriptor.
  • an acquisition module configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a binarization of the retrieval descriptor data
  • receiving the visual information extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the
  • the decompression module is further configured to: decompress the compressed data of the Q 3D map descriptors at the kth level to obtain Q 3D map descriptions Reconstructed data of the child; according to part or all of the data of the search descriptor, perform r-th level retrieval in the reconstructed data of the Q 3D map descriptors to obtain the N 3D map descriptors;
  • N ⁇ Q ⁇ P the decompression process of the compressed data of the P 3D map descriptors includes the m-level decompression and the k-level decompression, r and k are positive integers, m ⁇ k , j ⁇ r ⁇ L.
  • the compressed data of the P 3D map descriptors includes the respective binarized data and quantized data of the P 3D map descriptors
  • the decompression module is specifically configured to: Dequantize the respective quantized data of the descriptors to obtain P dequantized data, and the P dequantized data are used as reconstruction data of the P 3D map descriptors; each of the Q 3D map descriptors Quantized data of the quantized data is dequantized, and Q dequantized data are obtained; according to the Q dequantized data and the binarized data of the Q 3D map descriptors, the Q 3D map descriptors are respectively obtained. of reconstructed data.
  • the compressed data of the P 3D map descriptors includes the respective binarized data and quantized data of the P 3D map descriptors, and the decompression module, specifically It is used for: performing dequantization processing on the respective quantized data of the P 3D map descriptors to obtain P dequantized data; according to the respective binary values of the P dequantized data and the P 3D map descriptors to obtain the respective reconstruction data of the P 3D map descriptors.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the collection includes at least one 3D map descriptor
  • the retrieval module is used for: performing i-level retrieval in the S binarized data representing 3D map descriptors according to the binarized data of the retrieval descriptor, to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the embodiment of the present application provides a device for retrieving a 3D map.
  • the device may be a chip or a system-on-a-chip in an electronic device or a server, and may also be used in an electronic device or a server to implement the first aspect or the first aspect.
  • the retrieval device of the 3D map includes: a retrieval module, an extraction module and a decompression module.
  • the extraction module is configured to extract the first binarized data of the S 3D map descriptors from the compressed data of the S 3D map descriptors, and the S 3D map descriptors correspond to 3D A plurality of 3D map points in the map; a retrieval module, configured to perform i-th level retrieval in the first binarized data of the S 3D map descriptors according to the first binarized data of the retrieved descriptors, to Obtain P 3D map descriptors; the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device; the extraction module is also used to obtain the P 3D map descriptors Extract the second binarized data of the P 3D map descriptors from the compressed data; the retrieval module is used to describe the P 3D maps according to the second binarized data of the retrieved descriptors In order to obtain Q 3D map descriptors, S, P, Q, i and j are examples of the first bin
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors.
  • Point matching N is a positive integer, 0 ⁇ N ⁇ Q
  • the device also includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors , to obtain the pose information of the electronic device.
  • the i-th level retrieval and the j-th level retrieval adopt retrieval methods based on the first distance, and the P 3D map descriptors belong to the S 3D map descriptions
  • the position of the first binarized data of each 3D map descriptor of the P 3D map descriptors in the compressed data of the 3D map descriptor is different from the second binary value of the 3D map descriptor
  • the length of the first binarized data of each 3D map descriptor of the P 3D map descriptors is smaller than the length of the second binary data of each 3D map descriptor of the P 3D map descriptors.
  • the length of the valued data is smaller than the length of the first binarized data of each 3D map descriptor of the P 3D map descriptors.
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a first binary value of the retrieval descriptor. valued data and second binarized data; or, receive the visual information, extract the retrieval descriptor from the visual information, and perform binarization processing on the retrieval descriptor to obtain the retrieval The first binarized data and the second binarized data of the descriptor; or, in response to the operation of collecting visual information input by the user, triggering the sensor to collect visual information on the real environment to obtain the visual information, The retrieval descriptor is extracted from the visual information, and binarization is performed on the retrieval descriptor to obtain first binarized data and second binarized data of the retrieval descriptor.
  • an acquisition module configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a first binary value of the retrieval descriptor. valued data and second binarized data; or, receive the visual information,
  • the length of the first binarized data of the retrieval descriptor is equal to the length of the first binarized data of each of the S 3D map descriptors, and/or, The length of the second binarized data of the retrieval descriptor is equal to the length of the second binarized data of each 3D map descriptor of the S 3D map descriptors.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the set includes at least one 3D map descriptor
  • the retrieval module is specifically configured to: perform a search on the S first binarized data representing 3D map descriptors according to the first binarized data of the retrieved descriptor. Searching at the i-level to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the embodiment of the present application provides a device for retrieving a 3D map.
  • the device may be a chip or a system-on-a-chip in an electronic device or a server, and may also be used in an electronic device or a server to implement the first aspect or the first aspect.
  • the retrieval device of the 3D map includes: a retrieval module, an extraction module and a decompression module.
  • the decompression module is used to decompress the compressed data of the S 3D map descriptors at the mth level to obtain the first reconstructed data of the S 3D map descriptors, and the S 3D map descriptors
  • the map descriptor corresponds to multiple 3D map points in the 3D map
  • the retrieval module is used to perform the i-th level in the first reconstruction data of the S 3D map descriptors according to part or all of the data of the retrieval descriptor.
  • the retrieval descriptor is extracted from the visual information collected by the sensor of the electronic device and corresponds to the feature of the real environment; the decompression module is also used for the P
  • the compressed data of the 3D map descriptors is decompressed at the kth level to obtain the second reconstructed data of the P 3D map descriptors, and the decompression process of the compressed data of the S 3D map descriptors includes the mth stage decompression and the kth stage decompression, or, the decompression process of the compressed data of the S 3D map descriptors includes the kth stage decompression, and the kth stage decompression includes the mth stage decompression Compression; a retrieval module, configured to perform j-th level retrieval in the second reconstructed data of the P 3D map descriptors according to part or all of the retrieval descriptors, so as to obtain Q 3D map descriptors, S , P, Q, i
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors.
  • Point matching N is a positive integer, 0 ⁇ N ⁇ Q
  • the device also includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors , to obtain the pose information of the electronic device.
  • the i-th level retrieval and the j-th level retrieval adopt a retrieval method based on the second distance
  • the P 3D map descriptors belong to the S 3D map Descriptor
  • the distortion degree of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors, and the distortion of the second reconstructed data of each 3D map descriptor of the P 3D map descriptors degree is different
  • the distortion degree of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors is used to represent the first reconstructed data of each 3D map descriptor and the corresponding original 3D map descriptor
  • the degree of difference between the second reconstruction data of each 3D map descriptor of the P 3D map descriptors is used to represent the second reconstruction data of each 3D map descriptor and the corresponding original 3D
  • the distortion degree of the first reconstruction data of each 3D map descriptor of the P 3D map descriptors is greater than the second reconstruction data of each 3D map descriptor of the P 3D map descriptors Distortion of structured data.
  • the device further includes: an obtaining module, configured to: receive the retrieval descriptor, and obtain part or all of the data of the retrieval descriptor; or receive the visual information, and obtain the Extract the retrieval descriptor from the visual information, and obtain part or all of the data of the retrieval descriptor; or, in response to the operation of collecting visual information input by the user, trigger the sensor to perform visual information on the real environment Collecting, obtaining the visual information, extracting the retrieval descriptor from the visual information, and obtaining part or all of the data of the retrieval descriptor.
  • an obtaining module configured to: receive the retrieval descriptor, and obtain part or all of the data of the retrieval descriptor; or receive the visual information, and obtain the Extract the retrieval descriptor from the visual information, and obtain part or all of the data of the retrieval descriptor; or, in response to the operation of collecting visual information input by the user, trigger the sensor to perform visual information on the real environment Collecting, obtaining the visual information, extracting the retrieval de
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the collection includes at least one 3D map descriptor
  • the retrieval module is specifically configured to: perform the i-th reconstruction data in the m-th reconstructed data of the S representative 3D map descriptors according to part or all of the data of the retrieval descriptor. Level retrieval to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the embodiment of the present application provides a device for retrieving a 3D map.
  • the device may be a chip or a system-on-a-chip in an electronic device or a server, and may also be used in an electronic device or a server to implement the first aspect or the first aspect.
  • the retrieval device of the 3D map includes: a retrieval module, an extraction module and a decompression module.
  • the decompression module is used to decompress the compressed data of the S 3D map descriptors at the mth level to obtain the reconstructed data of the S 3D map descriptors, and the S 3D map descriptors Corresponding to multiple 3D map points in the 3D map;
  • the retrieval module is used to perform i-level retrieval in the reconstructed data of the S 3D map descriptors according to the partial data or all data of the retrieval descriptors, so as to obtain P A 3D map descriptor;
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device;
  • the extraction module is used to extract from the compressed data of the P 3D map descriptors The binarized data of the P descriptors;
  • the retrieval module is further configured to, according to the binarized data of the retrieved descriptors, perform j-th Level retrieval to get Q 3D map descriptors, S, P, Q
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors.
  • Point matching N is a positive integer, 0 ⁇ N ⁇ Q
  • the device also includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors , to obtain the pose information of the electronic device.
  • the retrieval method adopted by the i-th level retrieval is a retrieval method based on the second distance
  • the retrieval method adopted by the j-th level retrieval is a retrieval method based on the first distance
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a binarization of the retrieval descriptor data; or, receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor; or , in response to the operation of collecting visual information input by the user, triggering the sensor to collect visual information on the real environment, obtaining the visual information, extracting the retrieval descriptor from the visual information, and performing the retrieval
  • the descriptor is subjected to binarization processing to obtain binarized data of the retrieval descriptor.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data of the at least one data set
  • the collection includes at least one 3D map descriptor
  • the retrieval module is specifically configured to: perform the i-th reconstruction data in the m-th reconstructed data of the S representative 3D map descriptors according to part or all of the data of the retrieval descriptor. Level retrieval to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the embodiment of the present application provides a device for retrieving a 3D map, including: one or more processors; a memory for storing one or more programs; when the one or more programs are used by the one or more A plurality of processors are executed, so that the one or more processors implement the method described in any one of the first to fourth aspects or any possible design of any one of the first to fourth aspects.
  • an embodiment of the present application provides a computer-readable storage medium, which is characterized by comprising a computer program, and when the computer program is executed on a computer, the computer executes any one of the first to fourth aspects or The method described in any possible design of any one of the first to fourth aspects.
  • the embodiment of the present application provides a computer program or computer program product, when the computer program or computer program product is executed on the computer, the computer can realize any one of the first to fourth aspects and any one of them The method described in one possible implementation.
  • FIG. 1 is a schematic diagram of an application architecture provided by an embodiment of the present application
  • FIG. 2 is a schematic structural diagram of an electronic device 20 provided in an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a server 30 provided in an embodiment of the present application.
  • FIG. 4g is a schematic diagram of a user interface (such as a user interface of a 3D map application) displayed by an electronic device provided in an embodiment of the present application;
  • FIG. 5A to FIG. 5D are schematic flowcharts of fragments of a 3D map retrieval method provided by an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a 3D map retrieval method provided by an embodiment of the present application.
  • FIG. 7 is a schematic flowchart of a 3D map retrieval method provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a 3D map retrieval method provided by an embodiment of the present application.
  • Fig. 9 is a schematic diagram of the processing process of a 3D map retrieval method provided by the embodiment of the present application
  • FIG. 10 is a schematic structural diagram of a 3D map retrieval device provided by an embodiment of the present application.
  • FIG. 11 is a schematic block diagram of a decoding device 1100 provided by an embodiment of the present application.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • Fig. 1 is a schematic diagram of an application architecture provided by an embodiment of the present application.
  • the application architecture includes a plurality of electronic devices and servers, wherein the plurality of electronic devices may include a first electronic device, one or more A second electronic device (two second electronic devices are taken as an example in FIG. 1), the one or more second electronic devices are several electronic devices other than the first electronic device. Multiple electronic devices and servers, and multiple electronic devices can communicate with each other.
  • wireless-fidelity Wireless-fidelity
  • WiFi wireless-fidelity
  • Bluetooth Bluetooth
  • cellular 2/3/4/5 generation 2/3/4/5generation, 2G/3G/4G/5G
  • future communication methods which are not specifically limited.
  • one or more second electronic devices in the embodiment of the present application is only used to refer to other electronic devices except the first electronic device, but does not limit whether the types of multiple electronic devices are the same.
  • the above-mentioned electronic equipment can be various types of equipment equipped with cameras and display components.
  • the electronic equipment can be terminal equipment such as mobile phones, tablet computers, notebook computers, video recorders (in FIG. 1, the electronic equipment is a mobile phone as an example),
  • Electronic devices can also be devices used for virtual scene interaction, including VR glasses, AR devices, MR interactive devices, etc.
  • Electronic devices can also be wearable electronic devices such as smart watches and smart bracelets.
  • Electronic devices can also be vehicles, wireless Equipment carried in vehicles such as human-driven vehicles, drones, and industrial robots. The embodiment of the present application does not specifically limit the specific form of the electronic device.
  • the above-mentioned electronic devices may also be referred to as user equipment (user equipment, UE), subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station , terminal device, access terminal, mobile terminal, wireless terminal, smart terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.
  • user equipment user equipment
  • UE user equipment
  • subscriber station mobile unit
  • subscriber unit wireless unit
  • remote unit mobile device
  • wireless device wireless device
  • wireless communication device remote device
  • mobile subscriber station terminal device, access terminal, mobile terminal, wireless terminal, smart terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.
  • the above-mentioned servers may be one or more physical servers (a physical server is taken as an example in FIG. 1 ), may also be a computer cluster, or may be a virtual machine or a cloud server in a cloud computing scenario, and the like.
  • the electronic device can install virtual scene applications (applications, APPs) such as VR applications, AR applications, or MR applications, and can run VR applications based on user operations (such as clicking, touching, sliding, shaking, voice control, etc.).
  • application, AR application or MR application can collect the visual information of any object in the environment through the sensor, and then display the virtual object on the display component according to the collected visual information.
  • the virtual object can be a virtual object in a VR scene, an AR scene or an MR scene (that is, a objects).
  • the electronic device can install navigation, detection, and control application programs, and run corresponding applications based on user manipulation or preset programs.
  • Electronic devices can perform applications such as path planning, object detection, and vehicle control based on their own pose and other state information in the current environment.
  • the pose is the position and orientation information of the electronic device, which can be an absolute pose in the world coordinate system or a relative pose relative to a certain point in the environment.
  • the visual information involved in the embodiments of the present application includes, but is not limited to, images and videos collected by cameras (without depth information), images and videos with depth information collected by depth sensors, and laser radar (LiDAR) acquisitions.
  • the virtual scene application program in the electronic device may be an application program built in the electronic device itself, or an application program provided by a third-party service provider installed by the user himself, and no specific details are given here. limited.
  • the electronic device can also be equipped with a simultaneous localization and mapping (SLAM) system.
  • the SLAM system can create a map in a completely unknown environment, and use the map to perform positioning, pose (position and attitude) determination, navigation, etc.
  • the map created by the SLAM system is called a SLAM map.
  • the SLAM map can be understood as a map drawn by the SLAM system based on the environmental information collected by the collection device.
  • the collection device may include a visual information collection device in an electronic device and Inertial measurement unit (inertial measurement unit, IMU), wherein, the visual information acquisition device may include, for example, a camera, camera, depth camera, lidar, millimeter-wave radar, etc., and the IMU may include, for example, sensors such as a gyroscope and an accelerometer.
  • IMU Inertial measurement unit
  • the SLAM map is also referred to as a 3D map. It should be noted that the 3D map includes but is not limited to a SLAM map, and may also include a 3D map created using other technologies, which is not specifically limited in this embodiment of the present application.
  • the 3D map may include multiple 3D map points, and correspondingly, data of the 3D map may include data of multiple 3D map points.
  • a 3D map point is an interesting or salient feature point in the environment.
  • 3D map points are to use laser radar, aerial photography from the perspective of drones (oblique photography), high-definition panoramic cameras, high-definition industrial cameras and other equipment to shoot, through ORB, scale-invariant feature transformation ( scale-invariant feature transform, SIFT), accelerated version has robust features (speeded up robust features, SURF), binary robust independent elementary features (binary robust independent elementary features, BRIEF), binary robust invariant scalable key Point (binary robust invariant scalable keypoints, BRISK), fast retina keypoint (fast retina keypoint, FREAK), D2Net or self-supervised training-based feature point detection and descriptor extraction method (SuperPoint) and other methods from the data captured by the above equipment extract from.
  • scale-invariant feature transformation scale-invariant feature transform, SIFT
  • accelerated version has robust features (speeded up robust features, SURF), binary robust independent elementary features (binary robust independent elementary features, BRIEF), binary robust invariant scalable key Point (binary robust invariant scalable
  • the data of 3D map points can include:
  • the 3D map point descriptor is a vector (vector), which is used to represent the local features of the corresponding 3D map point.
  • the 3D map point descriptor is used for matching between 3D map points.
  • One possible approach is to calculate the distance between two 3D map point descriptors (it can be Euclidean distance, inner product distance, Hamming distance, etc.), and when the distance is less than a threshold, it is considered that two 3D map points match .
  • the spatial position of a 3D map point can be represented by X, Y, and Z on the axis of the three-dimensional space, it can also be represented by latitude and longitude, and altitude, and it can also be represented by polar coordinates.
  • the embodiment of the present application expresses the spatial position of a 3D map point No specific restrictions are made.
  • the spatial position of the 3D map point can be the absolute position of the 3D map point, or the relative position of the 3D map point. For example, taking the center position of the entire area as the origin, all the spatial positions of the 3D map point are relative to the spatial position of the origin offset position.
  • each 3D map point can be assigned a number and written into the data of the 3D map, and the storage order of multiple 3D map points in the memory can also be used to implicitly represent the number of the 3D map point.
  • the order of multiple 3D map points contained in a 3D map has no practical significance, so the aforementioned numbers can be considered as identifiers for identifying 3D map points to distinguish each 3D map point, but this Numbering is not used to limit the order of multiple 3D map points.
  • a 3D map contains 3 3D map points, and their numbers are 1, 2, and 3 respectively.
  • the processing of these 3 3D map points can be done according to The order of 1, 2, 3 can also be performed in the order of 3, 2, 1, and the order of 2, 1, 3 can also be performed, and so on.
  • the data of the 3D map further includes a plurality of region descriptors, and any one of the region descriptors in the plurality of region descriptors is used to describe part or all of the 3D map in the plurality of 3D map points
  • the feature of the point that is, for any one of the multiple area descriptors, the area descriptor can be used to describe the features of some or all of the 3D map points in the multiple 3D map points, so that the area descriptor and the 3D map A point is a one-to-many relationship.
  • each 3D map point in multiple 3D map points can be described by some or all of the multiple area descriptors, so the relationship between 3D map points and area descriptors is one-to-many. It can be seen that there is a many-to-many relationship between multiple region descriptors and multiple 3D map points.
  • the generation methods of regional descriptors include, but are not limited to, traditional methods such as bag of words (BOW), vector of locally aggregated descriptors (VLAD), and methods based on NetVLAD and artificial intelligence (AI). new method.
  • BOW bag of words
  • VLAD vector of locally aggregated descriptors
  • AI artificial intelligence
  • multiple region descriptors may also be identified with a number to distinguish the multiple region descriptors, but the number is also not a limitation on the order of the multiple region descriptors.
  • the data of the 3D map also includes the correspondence between the 3D map points and the descriptors, which clearly describe which 3D map points any one descriptor corresponds to, and any 3D map Which descriptors dots correspond to.
  • the above correspondence can be explicitly described by a correspondence table between the number of the region descriptor and the number of the 3D map point.
  • the 3D map contains 3 region descriptors, the numbers are T1 ⁇ T3, and there are 6 3D map points, where the numbers of the spatial positions of the six 3D map points are P 1 to P 6 , and the numbers of the six 3D map point descriptors are F 1 to F 6 , and the corresponding table is shown in Table 1.
  • Table 1 is an example of the correspondence table between the number of the region descriptor and the number of the 3D map point, and the correspondence table can also be presented in other formats or ways, which is not specifically limited in this application.
  • the above corresponding relationship can also be implicitly described by the storage location of the region descriptor and the 3D map point, for example, first store T1 in the memory, then store the data of P 1 , P 2 and P 3 , and then store T2, followed by storing the data of P2 and P3 , and finally storing T3, followed by storing the data of P3 , P4 , P5 and P6 .
  • FIG. 2 is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application.
  • the electronic device 20 may be the first electronic device and one or more second electronic devices in the embodiment shown in FIG. 1 at least one of the .
  • the structure shown in FIG. 2 does not constitute a specific limitation on the electronic device 20 .
  • the electronic device 20 may include more or fewer components than the structure shown in FIG. 2 , or combine certain components, or separate certain components, or arrange different components.
  • the various components shown in Figure 2 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
  • the electronic device 20 may include: a chip 21, a memory 22 (one or more computer-readable storage media), a user interface 23, a display component 24, a camera 25, a sensor 26, a positioning module 27 for positioning the device, and a communication device. Transceiver 28. Communication between these components may be via one or more buses 29 .
  • the chip 21 may include: one or more processors 211 , a clock module 212 and a power management module 213 .
  • the clock module 212 integrated in the chip 21 is mainly used to provide the processor 211 with a timer required for data transmission and timing control, and the timer can realize the clock function of data transmission and timing control.
  • the processor 211 can perform operations according to instruction opcodes and timing signals, generate operation control signals, and complete control of fetching and executing instructions.
  • the power management module 213 integrated in the chip 21 is mainly used to provide stable and high-precision voltage for the chip 21 and other components of the electronic device 20 .
  • the processor 211 may also be referred to as a central processing unit (central processing unit, CPU), and the processor 211 may specifically include one or more processing units, for example, the processor 211 may include an application processor (application processor, AP), modem Tuning processor, graphics processing unit (graphics processing unit, GPU), image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor , and/or, a neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem Tuning processor graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • NPU neural network processing unit
  • different processing units may be independent devices, or may be integrated in one or more processors.
  • the processor 211 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or, a universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the memory 22 can be connected to the processor 211 through the bus 29, or can be coupled with the processor 311, and is used for storing various software programs and/or multiple sets of instructions.
  • the memory 22 may include a high-speed random access memory (such as a cache memory), or may include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices or other non-volatile solid-state storage devices.
  • the memory 22 can store operating systems, for example, Android (Android), Apple Mobile Platform (IOS), Microsoft Window Operating System (Windows) or embedded operating systems such as Linux.
  • the memory 22 can also store data, for example, image data, point cloud data, 3D map data, pose data, coordinate system conversion information, map update information, and the like.
  • the memory 22 may also store computer-executable program codes, where the computer-executable program codes include instructions, for example, communication program instructions, related program instructions of the SLAM system, and the like.
  • the memory 22 can also store one or more application programs, for example, virtual scene application programs such as AR/VR/MR, map application programs, image management application programs, navigation and control application programs, and the like.
  • the memory 22 can also store a user interface program.
  • the user interface program can display the content of the application program through a graphical operation interface, for example, virtual objects in virtual scenes such as AR/VR/MR. 24 presentation, and realize receiving the user's control operations on the application through input controls such as menus, dialog boxes, and buttons.
  • the user interface 23 may be, for example, a touch panel, and the touch panel may detect a user's operation instruction thereon, and the user interface 23 may also be, for example, a keypad, a physical button, or a mouse.
  • Electronic device 20 may include one or more display components 24 .
  • the electronic device 20 can jointly realize the display function through the display component 24, the graphics processing unit (GPU) and the application processor (AP) in the chip 21, etc.
  • the GPU is a microprocessor that implements image processing and is connected to the display assembly 24 and the application processor.
  • the GPU performs mathematical and geometric calculations for graphics rendering.
  • the display component 24 can display the interface content output by the electronic device 20, for example, display images, videos, etc. in virtual scenes such as AR/VR/MR, and the interface content can include the interface of the running application program and the system level menu, etc.
  • interface elements such as buttons (Button), text input boxes (Text), sliders (Scroll Bar), menus (Menu), etc.; output interface elements, such as windows (Window), labels (Label), image, video, animation, etc.
  • the display component 24 can be a display panel, glasses (such as VR glasses), projection screen and so on.
  • the display panel may also be called a display screen, for example, may be a touch screen, a flexible screen, a curved screen, etc., or may be other optical components.
  • the display screen of the electronic device in the embodiment of the present application may be a touch screen, flexible screen, curved screen or other forms of screen, that is, the display screen of the electronic device has the function of displaying images, and the specific The material and shape are not specifically limited.
  • the display panel can be a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), an active matrix organic light-emitting diode or an active matrix Organic Light Emitting Diode (active-matrix organic light emitting diode, AMOLED), flexible light emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED )Wait.
  • liquid crystal display liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED active matrix organic light-emitting diode
  • FLED flexible light emitting diode
  • Miniled MicroLed, Micro-oLed
  • quantum dot light emitting diodes quantum dot light emitting diodes (quantum dot light emitting diodes, QLED )Wait.
  • the touch panel in the user interface 23 and the display panel in the display component 24 can be coupled together, for example, the touch panel can be arranged under the display panel, and the touch panel is used for Detecting the touch pressure on the display panel when the user inputs a touch operation (such as clicking, sliding, touching, etc.) through the display panel, and the display panel is used for displaying content.
  • a touch operation such as clicking, sliding, touching, etc.
  • the camera 25 may be a monocular camera, a binocular camera or a depth camera, and is used for shooting/recording the environment to obtain images/video images.
  • the image/video image collected by the camera 25 can be used as a kind of input data of the SLAM system, or can be displayed by the display component 24 for image/video.
  • the camera 25 can also be regarded as a sensor.
  • the image collected by the camera 25 may be in the IMG format or in other formats, which is not specifically limited in this embodiment of the present application.
  • the sensor 26 can be used to collect data related to state changes of the electronic device 20 (for example, rotation, swing, movement, shaking, etc.), and the data collected by the sensor 26 can be used as an input data of the SLAM system.
  • the sensor 26 may include one or more sensors, for example, an inertial measurement unit (inertial measurement unit, IMU), a time of flight (time of flight, TOF) sensor, and the like.
  • the IMU may include sensors such as a gyroscope and an accelerometer.
  • the gyroscope is used to measure the angular velocity of the electronic device when it is in motion
  • the accelerometer is used to measure the acceleration of the electronic device when it is in motion.
  • the TOF sensor can include a light emitter and a light receiver.
  • the light emitter is used to emit light, such as laser, infrared, radar waves, etc.
  • the light receiver is used to detect reflected light, such as reflected laser, infrared, and radar waves. Wait.
  • the sensor 26 may also include more other sensors, such as inertial sensors, barometers, magnetometers, wheel speedometers, etc., which are not specifically limited in this embodiment of the present application.
  • the positioning module 27 is used to realize the physical positioning of the electronic device 20 , for example, to acquire the initial position of the electronic device 20 .
  • the positioning module 27 may include one or more of a WiFi positioning module, a Bluetooth positioning module, a base station positioning module, and a satellite positioning module.
  • the satellite positioning module can be provided with a global navigation satellite system (global navigation satellite system, GNSS) to assist positioning, and GNSS is not limited to the Beidou system, the global positioning system (global positioning system, GPS) system, the GLONASS (global navigation satellite system) , GLONASS) system, Galileo satellite navigation system (Galileo) system.
  • the transceiver 28 is used to enable communication between the electronic device 20 and other devices (eg, servers, other electronic devices, etc.).
  • the transceiver 28 integrates a transmitter and a receiver for sending and receiving radio frequency signals, respectively.
  • the transceiver 28 includes, but is not limited to: an antenna system, a radio frequency (radio frequency, RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec ( CODEC) chip, subscriber identification module (SIM) card and storage media, etc.
  • the transceiver 28 can also be implemented on a separate chip.
  • the transceiver 28 supports at least one data network communication in 2G/3G/4G/5G, etc., and/or supports at least one of the following short-range wireless communication methods: Bluetooth (bluetooth, BT) communication, wireless fidelity (wireless fidelity, WiFi) communication, near field communication (near field communication, NFC), infrared (infrared, IR) wireless communication, ultra wide band (ultra wide band, UWB) communication, ZigBee protocol (ZigBee) communication.
  • Bluetooth bluetooth, BT
  • wireless fidelity wireless fidelity, WiFi
  • NFC near field communication
  • infrared infrared
  • UWB ultra wide band
  • ZigBee protocol ZigBee protocol
  • the processor 211 executes various functional applications and data processing of the electronic device 20 by running the program code stored in the memory 22 .
  • FIG. 3 is a schematic structural diagram of a server 30 provided in an embodiment of the present application.
  • the server 30 may be the server in the embodiment shown in FIG. 1 .
  • Server 30 includes processor 301 , memory 302 (one or more computer-readable storage media), and transceiver 303 . Communication between these components may be via one or more buses 304 .
  • the processor 301 may be one or more CPUs. In the case where the processor 301 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.
  • the memory 302 can be connected to the processor 301 through the bus 304, or can be coupled with the processor 301, and is used to store various program codes and/or multiple sets of instructions, and data (eg, map data, pose data, etc.).
  • the memory 302 includes, but is not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM), or portable read-only memory (Compact Disc Read-Only Memory, CD-ROM), etc.
  • the transceiver 303 mainly integrates a receiver and a transmitter, where the receiver is used to receive data sent by the electronic device (for example, requests, images, etc.), and the transmitter is used to send data to the electronic device (for example, map data, pose data, etc.) .
  • server 30 shown in FIG. 3 is only an example provided by the embodiment of the present application, and the server 30 may have more components than shown in the figure, which is not specifically limited in the embodiment of the present application.
  • the processor 301 executes various functional applications and data processing of the server 30 by running the program code stored in the memory 302 .
  • Coupled used in the embodiments of the present application means to be directly connected or connected through one or more intermediate components or circuits.
  • Figure 4a is a schematic diagram of an application scenario provided by the embodiment of the present application.
  • the application scenario is that the electronic device collects visual information through a sensor, and determines the current pose of the electronic device in combination with the visual information and the 3D map from the server.
  • the 3D map is provided by the server, that is, the server creates the 3D map, then compresses the 3D map, and transmits the compressed data of the 3D map to the electronic device; the electronic device performs decompression processing after receiving the compressed data of the 3D map, and obtains
  • the reconstruction data of the 3D map is combined with the collected visual information and the reconstruction data of the 3D map to determine the current pose of the electronic device.
  • the pose is the position information of the electronic device, which can be the absolute pose in the world coordinate system, or the relative pose relative to a certain point in the environment.
  • the server may pre-create the 3D map, compress the 3D map and store it locally, which can save storage space.
  • the server can also transmit the compressed data of the 3D map to other devices, such as cloud storage.
  • the server creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
  • Compressing the 3D map by the server can save local storage space.
  • the electronic device sends a map download request to the server. There are two ways to trigger the map download request:
  • the user opens the map application program installed on the electronic device, and the application program uploads the location information obtained based on GPS positioning or WiFi positioning to its corresponding server, and the uploading operation can trigger a map download request. Since the uploaded content includes location information, the server can make a preliminary estimate based on the location information, and transmit the compressed data of the 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device.
  • the scope of the area to which the anchor point indicated by the location information can be preset.
  • the area to which the anchor point belongs can be the administrative region of each level (including counties, cities, countries, or administrative districts, etc.) where the anchor point is located, or it can be centered on the anchor point , Set the distance as the circle area of the radius.
  • the user opens the map application program installed on the electronic device, and actively inputs or selects an area in the application program, for example, the user actively enters "xx business center", or selects from a list "A street, B street, "A Street” was selected in "C Street”.
  • the foregoing operations of the user may trigger a map download request.
  • Either the user input or selection includes a geographic location, so the server transmits the compressed data of the 3D map of the geographic location to the electronic device.
  • the electronic device automatically detects whether it meets the conditions for downloading a 3D map or starts downloading a 3D map, or the electronic device passes the detection A change in ambient light or a change in the environment is detected to start downloading the 3D map, so as to request the server to download a 3D map within an area, and the size of the area is not specifically limited.
  • the server sends the compressed data of the 3D map to the electronic device.
  • Electronic equipment collects visual information through sensors.
  • step 3 and step 4 are independent of each other, and the sequence is not limited.
  • the electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • the electronic device After receiving the compressed data of the 3D map, the electronic device does not need to decompress it immediately, but only needs to decompress to obtain the reconstruction data of the 3D map before positioning based on the visual information. For example, the user can pre-download the compressed data of an area-wide 3D map by downloading an "offline map", and decompress the compressed data of the 3D map when positioning is required.
  • Figure 4b is a schematic diagram of an application scenario provided by the embodiment of the present application.
  • the application scenario is that the electronic device collects visual information through a sensor, and the server determines the current position of the electronic device in combination with the visual information from the electronic device and a 3D map. posture.
  • the 3D map is provided by the server, that is, the server creates the 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally.
  • the server performs decompression processing to obtain the reconstructed data of the 3D map, and combines the visual information and the reconstructed data of the 3D map to determine the current pose of the electronic device.
  • the server creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
  • Electronic devices collect visual information through sensors.
  • the electronic device sends the visual information to the server.
  • the server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the server compresses the 3D map to save storage space.
  • the server performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • the server sends the pose to the electronic device.
  • Fig. 4c is a schematic diagram of an application scenario provided by the embodiment of the present application.
  • the application scenario is that the electronic device collects visual information through a sensor, and determines the current pose of the electronic device in combination with the collected visual information and a 3D map.
  • the 3D map is provided by the electronic device, that is, the electronic device creates the 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally.
  • the electronic device performs decompression processing to obtain the reconstructed data of the 3D map, and combines the collected visual information and the reconstructed data of the 3D map to determine the current pose of the electronic device.
  • the electronic device creates a 3D map and compresses the compressed data of the 3D map to be stored locally.
  • Electronic equipment collects visual information through sensors.
  • the electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • Fig. 4d is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the application scenario is that the second electronic device collects visual information through a sensor, and determines the location of the second electronic device in combination with the visual information and the 3D map from the server. current pose.
  • the 3D map is created by the first electronic device, that is, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed data of the 3D map to the server, and the server sends the compressed data of the 3D map to the second
  • the second electronic device performs decompression processing to obtain the reconstruction data of the 3D map, and determines the current pose of the second electronic device in combination with the collected visual information and the reconstruction data of the 3D map.
  • the first electronic device may pre-create a 3D map, compress the 3D map and then transmit it to the server, which can reduce transmission bandwidth.
  • the first electronic device creates a 3D map and compresses it to obtain compressed data of the 3D map.
  • the first electronic device sends the compressed data of the 3D map to the server.
  • the first electronic device compresses the 3D map before transmitting it, which can reduce the transmission bandwidth and improve the transmission efficiency.
  • the second electronic device sends a map download request to the server.
  • the map download request sent by the second electronic device may also be based on the trigger manner shown in FIG. 4a.
  • the server sends the compressed data of the 3D map to the second electronic device.
  • the second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the second electronic device collects the visual information through the sensor.
  • the second electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • Fig. 4e is a schematic diagram of an application scenario provided by the embodiment of the present application.
  • the application scenario is that the second electronic device collects visual information through a sensor, and the server combines the visual information from the second electronic device with the visual information from the first electronic device.
  • the 3D map of the device determines the current pose of the second electronic device.
  • the 3D map is created by the first electronic device, that is, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed data of the 3D map to the server.
  • the server performs decompression processing to obtain the reconstructed data of the 3D map, and combines the visual information from the second electronic device and the reconstructed data of the 3D map to determine the current pose of the second electronic device.
  • the first electronic device creates a 3D map and compresses it to obtain compressed data of the 3D map.
  • the first electronic device sends the compressed data of the 3D map to the server.
  • the second electronic device acquires visual information through sensor collection.
  • the second electronic device sends a positioning request to the server, and the positioning request carries visual information.
  • the server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the server performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • the server sends the pose obtained by positioning to the second electronic device.
  • Fig. 4f is a schematic diagram of an application scenario provided by the embodiment of the present application.
  • the application scenario is that the second electronic device collects visual information through a sensor, and combines the visual information and the 3D map from the first electronic device to determine the second The current pose of the electronic device.
  • the 3D map is created by the first electronic device, that is, the first electronic device creates a 3D map, performs compression processing on the 3D map, and then sends the compressed data of the 3D map to the second electronic device, and the second electronic device performs decompression processing , obtain the reconstructed data of the 3D map, and determine the current pose of the second electronic device in combination with the collected visual information and the 3D map from the first electronic device.
  • the first electronic device creates a 3D map, and compresses the compressed data of the 3D map to be stored locally.
  • the second electronic device sends a map download request to the first electronic device.
  • the first electronic device sends the compressed data of the 3D map to the second electronic device.
  • the second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.
  • the second electronic device acquires visual information through sensor collection.
  • the second electronic device performs positioning in the 3D map according to the visual information to obtain a pose corresponding to the visual information.
  • the positioning algorithm used may include:
  • Extract the region descriptor to be retrieved from the visual information, and the algorithm used to extract the region descriptor to be retrieved is the same as the algorithm for extracting the region descriptor from the 3D map.
  • the distance between the region descriptor to be retrieved and each region descriptor in multiple region descriptors can be calculated, the distance can include Hamming distance, Manhattan distance or Euclidean distance, etc., and then select the matching condition (For example, at least one region descriptor whose distance is smaller than the threshold value) is a candidate region descriptor.
  • the pose solving algorithm calculates the pose of the electronic device.
  • FIG. 4g is a schematic diagram of a user interface displayed by the electronic device provided in the embodiment of the present application.
  • the electronic device can display the user interface shown in FIG. 4g based on the pose.
  • the user interface may include a navigation arrow indication for navigating to the conference room 2, and the navigation arrow indication for navigating to the conference room 2 may be a virtual object obtained from the server or locally based on the pose.
  • the user interface may also include visual information collected by sensors, for example, buildings as shown in FIG. 4g. The user goes to the conference room 2 referring to the user interface shown in FIG. 4g of the electronic device.
  • the reconstructed data of the 3D map obtained through decompression involved in the embodiment of the present application may also be referred to as the reconstructed data of the 3D map.
  • FIGS. 4a to 4f all involve compressing and decompressing the 3D map, and performing positioning based on the reconstructed data of the 3D map obtained through decompression, so as to obtain the current pose of the electronic device.
  • a feasible way is to perform positioning based on the reconstructed data of the 3D map obtained through complete decompression.
  • the 3D map retrieval method provided in the following embodiments of the present application may be used to filter out descriptors and use the descriptors for positioning.
  • the following embodiments of the present application provide a retrieval method for a 3D map, by alternately performing retrieval and decompression or extracting from compressed data, so as to perform multi-level retrieval in the 3D map and obtain descriptors.
  • a retrieval method for a 3D map by alternately performing retrieval and decompression or extracting from compressed data, so as to perform multi-level retrieval in the 3D map and obtain descriptors.
  • at least the other level of retrieval in the two-level retrieval can guarantee the retrieval accuracy.
  • Incomplete decompression refers to incomplete decompression to obtain all reconstructed data of the 3D map. A method for searching the 3D map will be described below.
  • the retrieval of the 3D map involved in the embodiment of the present application refers to finding some of the most similar or most relevant 3D map points among the multiple 3D map points of the 3D map.
  • the length of the binarized data involved in the embodiment of the present application may be the number of bits (also referred to as the number of bits) of the binarized data. For example, if the binarized data is 110, the number of bits of the binarized data is 3.
  • retrieval method provided in the embodiment of the present application can also be applied to other technical fields, for example, retrieval of image or video data, retrieval of audio data, retrieval of point cloud data, and the like.
  • FIG. 5A is a schematic flowchart of a fragment of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • the method includes but is not limited to the following steps:
  • S101A Extract binarized data of S 3D map descriptors from the compressed data of S 3D map descriptors, where the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the binarized data may be first binarized data, second binarized data, ... nth binarized data, etc., n is a positive integer. It should be understood that “first”, “second” and “third” in the first binarized data, the second binarized data and the third binarized data, etc. differentiate each other.
  • the n-th binarized data is taken as an example for explanation below.
  • the nth binarized data of the S 3D map descriptors is extracted from the compressed data of the S 3D map descriptors.
  • S is a positive integer.
  • S represents the number of 3D map descriptors that need to be retrieved in the i-th level retrieval.
  • S ⁇ T, T represents the total number of 3D map descriptors in the 3D map.
  • the number of multiple 3D map points corresponding to the S 3D map descriptors may be greater than S, or equal to S.
  • the S 3D map descriptors may be S area descriptors, or S 3D map point descriptors.
  • one region descriptor corresponds to multiple 3D map points
  • one 3D map point descriptor corresponds to one 3D map point.
  • the compressed data of the S 3D map descriptors may be obtained by compressing the S region descriptors or the S 3D map point descriptors.
  • the compression processing here may include: compaction processing.
  • the compaction processing may include binarization processing and quantization processing.
  • the nth binarized data of the S 3D map descriptors may be partial data of the respective compressed data of the S 3D map descriptors, for example, the nth binarized data of the S 3D map descriptors may be S 3D maps Descriptors are part or all of the binarized data of the respective compressed data.
  • the binarization process includes but is not limited to: iterative quantization (iterative quantization, ITQ) hashing, locality-sensitive hashing (locality-sensitive hashing, LSH), or spectral hashing (spectral hashing).
  • ITQ iterative quantization
  • LSH locality-sensitive hashing
  • spectral hashing spectral hashing
  • the descriptor can be mapped to Hamming space (also called binary space) through binarization processing to obtain binarized data.
  • the quantization process includes, but is not limited to: scalar quantization, vector quantization, or product quantization.
  • the 3D map descriptor may be compressed into one or more quantization indices through quantization processing, and the one or more quantization indices are quantized data of the 3D map descriptor. Wherein, each quantization index in the one or more quantization indices corresponds to a quantization center.
  • the execution subject of the embodiment of the present application may perform entropy decoding on the compressed data of the S 3D map descriptors, and extract n-th binarized data of the S 3D map descriptors therefrom.
  • n-th binarized data is taken as an example for explanation.
  • the i-th level search is performed on the n-th binarized data of the S 3D map descriptors to obtain P 3D map descriptors.
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device.
  • the retrieval descriptor here may be the region descriptor to be retrieved in the above embodiment, or the 3D map point descriptor to be retrieved. That is, the retrieval descriptor may be a region to be retrieved descriptor extracted from visual information, and the region to be retrieved descriptor is used to describe features of some or all of the 3D map points among the plurality of 3D map points.
  • the region descriptor to be retrieved may be a vector, for example, a 128-dimensional vector, or a 512-dimensional vector, and so on.
  • the dimensions of the region descriptor to be retrieved are not limited by the above examples, and may also be other values, which are not illustrated one by one in the embodiment of the present application.
  • the retrieval descriptor may be a 3D map point descriptor to be retrieved extracted from visual information, and the 3D map point descriptor to be retrieved is used to represent a local feature of the corresponding 3D map point.
  • the 3D map point descriptor to be retrieved may be a vector, and the dimension of the vector may be any positive integer.
  • the ways of obtaining the retrieval descriptor may include but not limited to the following ways.
  • the execution subject of this embodiment may receive the retrieval descriptor sent by other devices.
  • the execution subject of this embodiment may be the first electronic device as shown in FIG. 1
  • the first electronic device may receive the retrieval descriptor sent by the second electronic device.
  • the execution subject of this embodiment may be a server as shown in FIG. 1 , and the server may receive the retrieval descriptor sent by the first electronic device or the second electronic device.
  • the execution subject of this embodiment may receive visual information sent by other devices, and extract a retrieval descriptor from the visual information.
  • the execution subject of this embodiment may be the first electronic device as shown in FIG.
  • the first electronic device may receive the visual information sent by the second electronic device, and the first electronic device extracts the retrieval descriptor from the visual information.
  • the execution subject of this embodiment may be a server as shown in FIG. 1 , the server may receive the visual information sent by the first electronic device or the second electronic device, and the server extracts the retrieval descriptor from the visual information.
  • the execution subject of this embodiment can detect the operation of collecting visual information input by the user, and in response to the operation, trigger its own sensor to collect visual information from the real environment, obtain visual information, and extract and retrieve the visual information from the visual information. descriptor.
  • the execution subject of this embodiment may be the first electronic device as shown in FIG. 1.
  • the first electronic device When the first electronic device detects the operation of collecting visual information input by the user, in response to the operation, it triggers its own sensor to monitor the real environment.
  • the visual information is collected, the visual information is obtained, and the retrieval descriptor is extracted from the visual information.
  • the operation of collecting visual information may be to start the sensor to collect visual information by clicking, touching, sliding, or shaking, or to start the sensor to collect visual information by voice control or other means, which is not limited in this embodiment of the present application.
  • the first electronic device detects the user's touch operation on the shooting button on the user interface, in response to the touch operation, the camera of the first electronic device is activated and so on.
  • the nth binarized data of the search descriptor may be part or all of the binarized data of the search descriptor.
  • the execution subject of this embodiment can perform binarization processing on the retrieval descriptor to obtain the binarized data of the retrieval descriptor, and select (also called extraction) part or All as the nth binarized data of the retrieval descriptor.
  • the execution subject of this embodiment can use the nth binarization data of the retrieval descriptor to search in the nth binarization data of S 3D map descriptors, so as to obtain the nth binarization data of the retrieval descriptor The most similar or most related P 3D map descriptors.
  • P is a positive integer, and P represents the number of 3D map descriptors filtered out by the i-th level retrieval. P ⁇ T.
  • P P ⁇ S.
  • a 3D map descriptor set is obtained through the i-level retrieval, and the number of 3D map descriptors in the 3D map descriptor set is less than the 3D map descriptions in the 3D map description subset before the i-level retrieval the number of children.
  • P may be greater than S.
  • the similarity or correlation between the nth binarized data of each of the P 3D map descriptors and the nth binarized data of the retrieval descriptor degree which is higher than other 3D map descriptors in the S 3D map descriptors except the P 3D map descriptors.
  • there are many ways to calculate the similarity or correlation For example, by calculating the first distance of the respective nth binarized data of two 3D map descriptors (3D map descriptor and retrieval descriptor), determine Correlation of the nth binarized data of two 3D map descriptors. Wherein, the smaller the first distance is, the higher the correlation is.
  • the first distance may include a distance obtained by using binarized data, such as a Hamming distance.
  • the first distance may be the Hamming distance.
  • the first distance may be a distance obtained by taking the absolute value of the difference between corresponding bits of the two binarized data and adding all the absolute values of the difference.
  • the retrieval based on the first distance refers to calculating the first distance to determine the degree of correlation or similarity to filter the 3D map descriptors.
  • the reconstructed data may be the first reconstructed data, the second reconstructed data, . . . the mth reconstructed data, etc., where m is a positive integer. It should be understood that the "first”, “second” and “third” in the first reconstructed data, the second reconstructed data and the third reconstructed data, etc., have no order, but are only for distinguishing each other .
  • the m-th reconstructed data is taken as an example for explanation below.
  • P 3D map descriptors can be screened and obtained, and through S103A, the compressed data of the P 3D map descriptors can be decompressed at the mth level to obtain the mth reconstructed data of the P 3D map descriptors.
  • m is a positive integer.
  • the mth stage of decompression may include inverse quantization processing.
  • the m-th stage of decompression may be staged decompression (that is, a part of complete decompression), or it may be complete decompression.
  • the mth reconstructed data of the P 3D map descriptors may be obtained by decompressing part or all of the compressed data of the P 3D map descriptors at the mth level.
  • the mth reconstructed data of the P 3D map descriptors may be dequantized data obtained by performing an inverse quantization process on part or all of the compressed data of the P 3D map descriptors.
  • the dequantized data may also be referred to as reconstructed data, and the reconstructed data may be obtained through dequantization methods in the prior art.
  • the mth reconstructed data of the P 3D map descriptors can be the quantized data (part of the compressed data) of the P 3D map descriptors, which are dequantized to obtain the P dequantized data, and then according to P pieces of dequantized data and P pieces of 3D map descriptors are obtained from binarized data (a part of compressed data).
  • the m-th stage decompression may also include other decompression processing such as entropy decoding and prediction, and this embodiment of the present application does not illustrate one by one.
  • Q is a positive integer
  • T represents the total number of 3D map descriptors in the 3D map
  • j i+1, 1 ⁇ i ⁇ L , 1 ⁇ j ⁇ L
  • L represents the total number of retrieval levels of the 3D map or a threshold value of the retrieval number
  • L is a positive integer greater than 1.
  • the reconstructed data may be the first reconstructed data, the second reconstructed data, . . . the mth reconstructed data, etc., where m is a positive integer. It should be understood that the "first”, “second” and “third” in the first reconstructed data, the second reconstructed data and the third reconstructed data, etc., have no order, but are only for distinguishing each other .
  • the m-th reconstructed data is taken as an example for explanation below.
  • Retrieving part or all of the data of the descriptor refers to retrieving part or all of the components of the descriptor.
  • the component of the retrieval descriptor refers to the value of a dimension in the vector of the retrieval descriptor.
  • the retrieval descriptor may be a vector, for example, an M-dimensional vector, each of which is a component of the vector.
  • the execution subject of this embodiment can use part or all of the data of the retrieval descriptor to search in the mth reconstruction data of the P 3D map descriptors to obtain the mth reconstruction data of the retrieval descriptor
  • the Q 3D map descriptors with the most similar or relevant data. Q can be smaller than P. It can also be understood that another 3D map descriptor set is obtained through the j-th level retrieval, and the number of 3D map descriptors in the 3D map descriptor set is less than the number of 3D map descriptors obtained by the i-th level retrieval.
  • the similarity or correlation between the mth reconstructed data of each of the Q 3D map descriptors and some or all of the data of the retrieval descriptor is higher than that of the P 3D map descriptors except for the Q 3D map descriptors Other 3D map descriptors.
  • there are many ways to calculate the similarity or correlation for example, by calculating the second of two 3D map descriptors (the mth reconstructed data of the 3D map descriptor and part or all of the data of the retrieval descriptor).
  • Distance which determines the degree of correlation between two 3D map descriptors.
  • the second distance may include, but is not limited to, Euclidean distance, inner product distance, cosine distance, Manhattan distance, and the like. For example, the smaller the Euclidean distance, the higher the correlation, and the larger the inner product distance, the higher the correlation.
  • condition judgment step can be performed after S104A.
  • the search of this embodiment can be ended, and if the condition is not met, one-level or multi-level search can be continued.
  • This condition can be any condition, which can be reasonably set according to requirements.
  • the condition can be that the similarity or correlation is higher than or equal to the preset threshold, or that the number of retrieved 3D map descriptors is less than or equal to the preset threshold. Set the number and so on.
  • the condition may be that the following N 3D map descriptors used for positioning are retrieved.
  • the retrieval method of the 3D map in the embodiment of the present application is a fragment process, which may also include other levels of retrieval before and/or after the fragment process.
  • the retrieval method of the 3D map may include more stages than those shown in FIG. 5A
  • the multi-level search for example, 3, 4 or 5 levels and so on. For example, there may be more levels of retrieval before S101A in the embodiment shown in FIG. 5A , or there may be more levels of retrieval after S104A in the embodiment shown in FIG. 5A , which will not be illustrated here one by one.
  • Q 3D map descriptors obtained through multi-level retrieval can be used for positioning. If the 3D map points corresponding to the N 3D map descriptors in the Q 3D map descriptors match the 3D map points corresponding to the retrieval descriptors, then positioning can be performed according to the 3D map points corresponding to the N 3D map descriptors to obtain Pose information of electronic devices.
  • the electronic device may be an electronic device that collects the aforementioned visual information. 0 ⁇ N ⁇ Q.
  • the binarized data of S 3D map descriptors are extracted from the compressed data of S 3D map descriptors, and the binarized data of S 3D map descriptors is In the i-level retrieval, P 3D map descriptors are screened. Perform m-level decompression on the compressed data of P 3D map descriptors to obtain the reconstructed data of P 3D map descriptors. The j-level retrieval is performed in the data to filter out a smaller number of 3D map descriptors.
  • the first-level retrieval of at least two-level retrieval uses binary data in the compressed data, which can increase the retrieval speed, and the other-level retrieval uses the reconstructed data obtained by decompression, which can ensure retrieval accuracy.
  • the retrieval method for the 3D map provided in the embodiment of the present application can improve the retrieval performance.
  • FIG. 5B is a schematic flow chart of a fragment of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • the method includes but is not limited to the following steps:
  • S101B Extract first binarized data of S 3D map descriptors from the compressed data of S 3D map descriptors, where the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the first binarized data may refer to any one of the first binarized data, the second binarized data, . . . the nth binarized data, and n is a positive integer. It should be understood that “first”, “second” and “third” in the first binarized data, the second binarized data and the third binarized data, etc. differentiate each other.
  • the n-th binarized data is taken as an example for explanation below.
  • the nth binarized data of the S 3D map descriptors are extracted from the compressed data of the S 3D map descriptors, and the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the n-th binarized data is taken as an example for explanation below.
  • the i-th level search can be performed on the nth binarized data of the S 3D map descriptors to obtain P 3D map descriptors.
  • S101B and S102B reference may be made to S101A and S102A of the embodiment shown in FIG. 5A .
  • P 3D map descriptors can be screened from the S 3D map descriptors, and then the qth binarization data of the P 3D map descriptors can be extracted from the compressed data of the P 3D map descriptors, To further perform retrieval based on the qth binarized data.
  • the qth binarized data of the P 3D map descriptors may be part or all of the compressed data of the P 3D map descriptors, for example, the qth binarized data of the P 3D map descriptors may be P Part or all of the binarized data for each of the 3D map descriptors.
  • the nth binarization data of each 3D map descriptor of the S 3D map descriptors is different from the qth binarization data of each 3D map descriptor of the P 3D map descriptors.
  • the difference may be different in length, that is, the length of the nth binarized data of each 3D map descriptor of the S 3D map descriptors is different from the length of each 3D map descriptor of the P 3D map descriptors. Different lengths.
  • the difference may be that the binarization processing methods corresponding to the nth binarized data and the qth binarized data are different, that is, the nth of each 3D map descriptor of the S 3D map descriptors
  • the binarization processing manner corresponding to the binarization data is different from the binarization processing manner corresponding to each 3D map descriptor of the P 3D map descriptors.
  • the nth binarized data of each 3D map descriptor of S 3D map descriptors is obtained through a binarization processing method (for example, locality-sensitive hashing), and each 3D data of P 3D map descriptors
  • the map descriptor is obtained through another binarization method (for example, spectral hashing).
  • the length of the nth binarized data can be measured or represented by the number of bits of the nth binarized data.
  • the length of the qth binarized data can be measured or represented by the number of bits of the qth binarized data.
  • the nth binarized data of each 3D map descriptor of P 3D map descriptors in the compressed data of 3D map descriptors The position is different from the position of the qth binarized data of the 3D map descriptor in the compressed data of the 3D map descriptor.
  • the location can be a start location or an end location, etc.
  • the component of the original 3D map descriptor corresponding to the nth binarization data of each 3D map descriptor of the P 3D map descriptors corresponds to the qth binarization data of the corresponding 3D map descriptor
  • the components of the original 3D map descriptor of are different.
  • the nth binarized data of a 3D map descriptor may be the binarized data of the first part of the 3D map descriptor (for example, the first 128-dimensional components of the 3D map descriptor), and the nth binarized data of the 3D map descriptor
  • the q binarization data may be the binarization data of the second part of the 3D map descriptor (for example, the last 128-dimensional components of the 3D map descriptor).
  • the length of the nth binarized data of each 3D map descriptor of P 3D map descriptors may be less than P 3D map descriptors
  • the length of the q-th binarized data of each 3D map descriptor of the child is relatively short, which can improve the retrieval speed, and the length of the qth binarized data of the 3D map descriptor in the j-level retrieval is longer. It can improve retrieval accuracy. Through at least two levels of differentiated retrieval, the comprehensive improvement of retrieval speed and retrieval accuracy can be realized.
  • the length of the nth binarized data of each 3D map descriptor of the P 3D map descriptors may be greater than the length of the qth binarized data of each 3D map descriptor of the P 3D map descriptors.
  • the length of the nth binarized data of each 3D map descriptor of the P 3D map descriptors may be equal to the length of the qth binarized data of each 3D map descriptor of the P 3D map descriptors.
  • the jth level retrieval is performed on the qth binarized data of the P 3D map descriptors to obtain Q 3D map descriptors.
  • the qth binarized data of the search descriptor may be part or all of the binarized data of the search descriptor.
  • the execution subject of this embodiment can perform binarization processing on the retrieval descriptor to obtain the binarized data of the retrieval descriptor, and select (also called extraction) part or All are represented as the n+1th of the retrieval descriptor. It should be noted that the nth binarized data of the retrieval descriptor and the qth binarized data of the retrieval descriptor may be the same or different.
  • the execution subject of this embodiment can use the qth binarized data of the search descriptor to search in the qth binarized data of the P 3D map descriptors, so as to obtain the qth binarized data of the search descriptor
  • the Q most similar or most relevant 3D map descriptors for the binarized data. Q can be smaller than P. It can also be understood that another 3D map descriptor set is obtained through the j-th level retrieval, and the number of 3D map descriptors in the 3D map descriptor set is less than the number of 3D map descriptors obtained by the i-th level retrieval.
  • the similarity or correlation between the qth binarized data of each of the Q 3D map descriptors and the qth binarized data of the retrieval descriptor is higher than that of the P 3D map descriptors except the Q 3D map descriptors other 3D map descriptors.
  • the explanation of the calculation method of the similarity degree or the correlation degree can refer to the relevant explanation explanation of S102B, which will not be repeated here.
  • the i-th level retrieval in the above S102B and the j-th level retrieval in the above S104B all use the retrieval method based on the first distance, and its specific explanation can refer to the S102A of the embodiment shown in FIG. 5A Explanation, not repeated here.
  • condition judgment step can be performed after S104B. If the condition is met, the search in this embodiment can be ended, and if the condition is not met, one or more levels of search can be continued.
  • This condition can be any condition, which can be reasonably set according to requirements.
  • the condition can be that the similarity or correlation is higher than or equal to the preset threshold, or that the number of retrieved 3D map descriptors is less than or equal to the preset threshold. Set the number, etc., the embodiment of the present application does not give examples one by one.
  • the retrieval method of the 3D map in the embodiment of the present application is a fragment process, which may also include other levels of retrieval before and/or after the fragment process.
  • the retrieval method of the 3D map may include more stages than those shown in FIG. 5B
  • the multi-level search for example, 3, 4 or 5 levels and so on. For example, there may be more levels of retrieval before S101B in the embodiment shown in FIG. 5B , or there may be more levels of retrieval after S104B in the embodiment shown in FIG. 5B , which will not be illustrated here one by one.
  • Q 3D map descriptors obtained through multi-level retrieval can be used for positioning. If the 3D map points corresponding to the N 3D map descriptors in the Q 3D map descriptors match the 3D map points corresponding to the retrieval descriptors, then positioning can be performed according to the 3D map points corresponding to the N 3D map descriptors to obtain Pose information of electronic devices.
  • the electronic device may be an electronic device that collects the aforementioned visual information. 0 ⁇ N ⁇ Q.
  • the first binarized data of S 3D map descriptors are extracted from the compressed data of S 3D map descriptors, and according to the first binarized data of the retrieved descriptors, the S 3D map descriptors are The i-level search is performed on the first binarized data, and P 3D map descriptors are obtained by screening. Extract the second binarized data of P 3D map descriptors from the compressed data of P 3D map descriptors, according to the second binarized data of the retrieved descriptors, in the second binary data of P 3D map descriptors The j-level retrieval is performed in the 2D data to filter out a smaller number of 3D map descriptors.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • FIG. 5C is a schematic flowchart of a fragment of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • the method includes but is not limited to the following steps:
  • S101C Decompress the compressed data of the S 3D map descriptors at the mth stage to obtain the first reconstructed data of the S 3D map descriptors, and the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the first reconstructed data may refer to any one of the first reconstructed data, the second reconstructed data, ... the mth reconstructed data, etc., m is a positive integer. It should be understood that the "first”, “second” and “third” in the first reconstructed data, the second reconstructed data and the third reconstructed data, etc., have no order, but are only for distinguishing each other . The following takes the first reconstructed data as the mth reconstructed data as an example for explanation.
  • S and m are positive integers.
  • S represents the number of 3D map descriptors that need to be retrieved in the i-th level retrieval.
  • S ⁇ T, T represents the total number of 3D map descriptors in the 3D map.
  • the number of multiple 3D map points corresponding to the S 3D map descriptors may be greater than S, or equal to S.
  • the compressed data of the S 3D map descriptors may be obtained by compressing the S region descriptors or the S 3D map point descriptors.
  • the compression processing here may include: quantization processing.
  • the compression processing may further include at least one of the following: binarization processing, prediction processing, or entropy coding processing.
  • the mth reconstructed data of the S 3D map descriptors is obtained by decompressing the partial data of the respective compressed data of the S 3D map descriptors in the mth stage.
  • the mth stage of decompression may include inverse quantization processing.
  • the mth stage of decompression may be a staged decompression (ie part of a full decompression).
  • the m-th reconstructed data of the S 3D map descriptors may be obtained by decompressing partial data of the respective compressed data of the S 3D map descriptors through m-th stage decompression.
  • the mth reconstructed data of the S 3D map descriptors may be dequantized data obtained by performing an inverse quantization process on parts of the compressed data of the S 3D map descriptors.
  • the dequantized data may also be referred to as reconstructed data, and the reconstructed data may be obtained through dequantization methods in the prior art.
  • the mth reconstructed data of the S 3D map descriptors may be a part of the quantized data (a part of the compressed data) of the S 3D map descriptors, which is dequantized to obtain the S dequantized data, and then It is obtained according to a part of binarized data (a part of compressed data) of S dequantized data and S 3D map descriptors.
  • the m-th stage decompression may also include other decompression processing such as entropy decoding and prediction, and this embodiment of the present application does not illustrate one by one.
  • the following takes the first reconstructed data as the mth reconstructed data as an example for explanation.
  • the i-th level retrieval is performed in the mth reconstructed data of the S 3D map descriptors to obtain P 3D map descriptors.
  • Retrieving part or all of the data of the descriptor refers to retrieving part or all of the components of the descriptor.
  • the component of the retrieval descriptor refers to the value of a dimension in the vector of the retrieval descriptor.
  • the retrieval descriptor may be a vector, for example, an M-dimensional vector, each of which is a component of the vector.
  • the execution subject of this embodiment can use part or all of the data of the search descriptor to search the mth reconstructed data of the S 3D map descriptors to obtain the most similar or most similar to the mth reconstructed data of the search descriptor The most relevant P 3D map descriptors.
  • P P ⁇ S.
  • a 3D map descriptor set is obtained through the i-level retrieval, and the number of 3D map descriptors in the 3D map descriptor set is less than the 3D map descriptions in the 3D map description subset before the i-level retrieval the number of children.
  • P may be greater than S.
  • the second distance may include, but is not limited to, Euclidean distance, inner product distance, cosine distance, Manhattan distance, and the like. For example, the smaller the Euclidean distance, the higher the correlation, and the larger the inner product distance, the higher the correlation.
  • the decompression process of the compressed data of the S 3D map descriptors includes the mth level Decompression and decompression at the kth stage, or, the decompression process of the compressed data of the S 3D map descriptors includes the kth stage of decompression, and the kth stage of decompression includes the mth stage of decompression.
  • the second reconstructed data may refer to any one of the second reconstructed data, the third reconstructed data, . . . the kth reconstructed data, k is a positive integer, and m ⁇ k. It should be understood that the "second" and "third" in the second reconstructed data, the third reconstructed data, etc. have no order, but are only for distinguishing each other. The following takes the second reconstructed data as the kth reconstructed data as an example for explanation.
  • P 3D map descriptors can be screened from S 3D map descriptors, and then the compressed data of the P 3D map descriptors can be decompressed at the kth level to obtain the kth of the P 3D map descriptors Restructure the data to further perform retrieval based on the kth reconstructed data.
  • k m+1.
  • the k-th reconstructed data of the P 3D map descriptors may be obtained by decompressing part or all of the compressed data of the P 3D map descriptors at the k-th stage.
  • both the i-level retrieval and the j-level retrieval adopt the retrieval method based on the second distance
  • the P 3D map descriptors belong to the S 3D map descriptors
  • the P 3D map descriptors The distortion degree of the mth reconstructed data of each 3D map descriptor of P is different from the distortion degree of the kth reconstructed data of each 3D map descriptor of P 3D map descriptors, each 3D map of P 3D map descriptors
  • the distortion degree of the mth reconstruction data of the descriptor is used to represent the degree of difference between the mth reconstruction data of each 3D map descriptor and the corresponding original 3D map descriptor, each 3D map description of P 3D map descriptors
  • the distortion degree of the kth reconstructed data of each sub is used to represent the degree of difference between the kth reconstructed data of each 3D map descriptor and the corresponding original 3D map descriptor, wherein, P ⁇ S
  • the distortion degree of the mth reconstruction data of each 3D map descriptor of the P 3D map descriptors is greater than the distortion degree of the kth reconstruction data of each 3D map descriptor of the P 3D map descriptors .
  • the distortion degree of the latter level of retrieval results in at least two levels of retrieval is smaller and smaller, thereby improving the accuracy of retrieval results.
  • P 3D map descriptors belong to S 3D map descriptors.
  • the 3D map descriptor The compressed data includes first quantized data and second quantized data, the first reconstructed data of the 3D map descriptor is obtained by decompressing the first quantized data at the first stage, and the first reconstructed data of the 3D map descriptor can be Including reconstructed data of partial components of the original vector of the 3D map descriptor, the second reconstructed data of the 3D map descriptor is obtained by decompressing the second quantized data at the second stage, the second reconstructed data of the 3D map descriptor
  • the compressed data includes first quantized data and second quantized data. Unlike the previous example, the second quantized data here can be obtained after quantizing the residual of the first quantized data.
  • the first quantized data of the 3D map descriptor The reconstructed data is obtained by decompressing the first quantized data at the first level, and the first reconstructed data of the 3D map descriptor may include the first reconstructed data of the original vector of the 3D map descriptor (reconstructed data with lower precision structural data), the second reconstructed data of the 3D map descriptor is obtained by decompressing the second quantized data at the second stage, and the second reconstructed data of the 3D map descriptor may include the above-mentioned residual of the 3D map descriptor
  • the following takes the second reconstructed data as the kth reconstructed data as an example for explanation.
  • the j-th level retrieval is performed in the k-th reconstructed data of the P 3D map descriptors to obtain Q 3D map descriptors.
  • S, P, Q, i, j, k, and m are positive integers, and 0 ⁇ Q ⁇ P, 0 ⁇ P ⁇ T, 0 ⁇ S ⁇ T, T represents the total number of 3D map descriptors in the 3D map
  • condition judgment step can be performed after S104C. If the condition is met, the search in this embodiment can be ended, and if the condition is not met, one or more levels of search can be continued.
  • This condition can be any condition, which can be reasonably set according to requirements.
  • the condition can be that the similarity or correlation is higher than or equal to the preset threshold, or that the number of retrieved 3D map descriptors is less than or equal to the preset threshold. Set the number, etc., the embodiment of the present application does not give examples one by one.
  • the retrieval method of the 3D map in the embodiment of the present application is a fragment process, which may also include other levels of retrieval before and/or after the fragment process.
  • the retrieval method of the 3D map may include more stages than those shown in FIG. 5C
  • the multi-level search for example, 3, 4 or 5 levels and so on. For example, there may be more levels of retrieval before S101C in the embodiment shown in FIG. 5C , or there may be more levels of retrieval after S104C in the embodiment shown in FIG. 5C , which are not illustrated here.
  • Q 3D map descriptors obtained through multi-level retrieval can be used for positioning. If the 3D map points corresponding to the N 3D map descriptors in the Q 3D map descriptors match the 3D map points corresponding to the retrieval descriptors, then positioning can be performed according to the 3D map points corresponding to the N 3D map descriptors to obtain Pose information of electronic devices.
  • the electronic device may be an electronic device that collects the aforementioned visual information. 0 ⁇ N ⁇ Q.
  • the compressed data of S 3D map descriptors is decompressed at the mth stage to obtain the first reconstructed data of S 3D map descriptors.
  • the i-th level search is performed on the first reconstructed data of the map descriptor, and P 3D map descriptors are screened out.
  • Decompress the compressed data of the P 3D map descriptors at the kth level to obtain the second reconstructed data of the P 3D map descriptors.
  • the j-th level retrieval is performed in the second reconstructed data to filter out a smaller number of 3D map descriptors.
  • a kind of reconstructed data of the 3D map descriptor obtained by decompression which is alternately performed with the first-level retrieval.
  • one level of retrieval is staged decompression, which can improve the retrieval speed, and the decompression differential settings of at least two levels of retrieval can ensure retrieval accuracy.
  • the degree of decompression or the degree of distortion of the reconstructed data of the 3D map descriptor used by any two levels of retrieval are different. Compared with retrieving in the reconstructed data of the fully decompressed 3D map, the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • FIG. 5D is a schematic flowchart of a fragment of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or also It can be applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • the method includes but is not limited to the following steps:
  • S101D Decompress the compressed data of the S 3D map descriptors at the mth level to obtain the reconstructed data of the S 3D map descriptors, and the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • the reconstructed data may refer to any one of the first reconstructed data, the second reconstructed data, ... the mth reconstructed data, etc., m is a positive integer. It should be understood that the "first”, “second” and “third” in the first reconstructed data, the second reconstructed data and the third reconstructed data, etc., have no order, but are only for distinguishing each other . The following takes the reconstructed data as the mth reconstructed data as an example for explanation.
  • the S 3D map descriptors correspond to multiple 3D map points in the 3D map.
  • S103D Extract binarized data of P 3D map descriptors from the compressed data of P 3D map descriptors.
  • the compressed data of S 3D map descriptors is decompressed at the mth level to obtain the reconstructed data of S 3D map descriptors, and according to the partial data or all data of the retrieval descriptors, the S 3D map descriptions
  • the i-th level retrieval is performed in the reconstructed data of the child, and P 3D map descriptors are screened out. Extract the binarized data of the P descriptors from the compressed data of the P 3D map descriptors, and perform jth in the binarized data of the P 3D map descriptors according to the binarized data of the retrieved descriptors.
  • Extraction or decompression from compressed data, and retrieval are performed alternately to perform multi-level retrieval in the 3D map to obtain a 3D map descriptor that can be used for positioning.
  • one level of retrieval is decompression, which can improve retrieval accuracy
  • one level of retrieval is extraction from compressed data, which can improve retrieval speed.
  • the retrieval method of the 3D map provided by the embodiment of the present application can improve the retrieval speed and ensure the retrieval accuracy.
  • retrieval method flows shown in FIG. 5A to FIG. 5C above can be flexibly combined with each other to form various multi-level retrieval methods.
  • FIG. 6 is a schematic flowchart of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or can also be Applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • This embodiment takes multi-level retrieval as an example of 4-level retrieval for illustration, wherein, the first-level retrieval and the second-level retrieval adopt the retrieval method based on the first distance, and the third-level retrieval and the fourth-level retrieval adopt the retrieval method based on the second distance. How to retrieve the distance.
  • the method includes but is not limited to the following steps:
  • the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device.
  • the S 1 3D map descriptors may be S 1 region descriptors, or S 1 3D map point descriptors. Wherein, one area descriptor corresponds to multiple 3D map points, and its specific explanation can refer to the explanation of the foregoing embodiments, and will not be repeated here.
  • the first binarized data of the retrieval descriptor may be partial binarized data of the retrieval descriptor.
  • the execution subject of this embodiment can perform binarization processing on the retrieval descriptor to obtain the binary data of the retrieval descriptor, and select a part from the binary data of the retrieval descriptor as the first part of the retrieval descriptor. Binarize data.
  • P 1 is smaller than S 1 . It can also be understood that a subset is obtained through retrieval, and the number of 3D map descriptors in this subset is less than the number of 3D map descriptors in the 3D map descriptor set before retrieval.
  • the bit overhead of the second binarized data in this embodiment may be greater than the bit overhead of the first binarized data, so that the retrieval speed of the previous stage of retrieval can be improved, and the accuracy of the subsequent stage of retrieval can be guaranteed.
  • the execution subject of this embodiment can use the second binarized data of the search descriptor to perform a search based on the first distance among the second binarized data of P 1 3D map descriptors, so as to obtain the The Q 1 D map descriptors most similar or most relevant to the second binarized data.
  • Q 1 is less than or equal to P 1 . It can also be understood that another subset is obtained through retrieval, and the number of 3D map descriptors in this subset is less than the number of 3D map descriptors in the 3D map descriptor subset retrieved in step 203 .
  • Q 2 is less than or equal to Q 1 .
  • another subset is obtained through retrieval, and the number of 3D map descriptors in this subset is less than the number of 3D map descriptors in the 3D map descriptor subset retrieved in step 205 .
  • the retrieval based on the second distance in the third stage can improve the retrieval accuracy.
  • Q 3 is less than or equal to Q 2 . It can also be understood that another subset is obtained through retrieval, and the number of 3D map descriptors in this subset is less than the number of 3D map descriptors in the 3D map descriptor subset retrieved in step 207 .
  • the binarized data of the 3D map descriptor is extracted from the compressed data or the reconstruction data of the 3D map descriptor is obtained by decompression, which is alternately performed with the first-level search, so as to perform multi-level search in the 3D map, and obtain A 3D map descriptor that can be used for localization.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • FIG. 7 is a schematic flowchart of a method for retrieving a 3D map provided by an embodiment of the present application.
  • the method can be applied to any of the electronic devices illustrated in FIG. 1 to FIG. 4f, or can also be Applied to the server shown in any one of Fig. 1 to Fig. 4f.
  • This embodiment is illustrated by taking the S 3D map descriptors in the embodiment shown in FIG. 5A as S representative 3D map descriptors.
  • the S representative 3D map descriptors each correspond to at least one data set.
  • the specific possible ways of 102A in the above-mentioned FIG. 5A may include but not limited to the following steps:
  • At least one representative 3D map descriptor corresponds to at least one data set, and each of the at least one data set includes compressed data of a plurality of 3D map descriptors.
  • the 3D map descriptors in each data set have certain correlation or similarity.
  • S represent the binarized data of the 3D map descriptor
  • the binarized data of the retrieval descriptor may be part or all of the binarized data of the retrieval descriptor.
  • the execution subject of this embodiment can use the binarized data of the search descriptor to perform a search based on the first distance among S binarized data representing 3D map descriptors, so as to obtain the binarized data corresponding to the search descriptor At least one of the most similar or most related represents a 3D map descriptor.
  • the number of at least one representative 3D map descriptor is less than or equal to S.
  • At least one data set is obtained by retrieving at least one representative 3D map descriptor, so as to obtain compressed data of P 3D map descriptors in the at least one data set.
  • the S representative 3D map descriptors are T 1 , T 2 and T 3 , T 1 corresponds to data set 1 , T 2 corresponds to data set 2 , and T 3 corresponds to data set 3 .
  • At least one representative 3D map descriptor obtained by searching is T 3 , thereby obtaining a data set 3 , which may include compressed data of P 3D map descriptors.
  • one-level retrieval is performed on a small number of representative 3D map descriptors to improve the retrieval speed, and then one-level or multi-level retrieval can be performed in the data set corresponding to the representative 3D map descriptor to improve retrieval accuracy Rate.
  • the retrieval speed can be improved and the retrieval accuracy can be guaranteed.
  • S 3D map descriptors in the embodiment shown in FIG. 5C to FIG. 5D are S representative 3D map descriptors, which is similar to the above-mentioned embodiment shown in FIG. 7 .
  • S representative 3D map descriptors which is similar to the above-mentioned embodiment shown in FIG. 7 .
  • FIG. 8 is a schematic flowchart of a method for retrieving a 3D map provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a processing procedure of a method for retrieving a 3D map provided by an embodiment of the present application. This embodiment includes a server and electronic equipment. The method includes but is not limited to the following steps:
  • the server determines the size relationship between each component of the S 3D map descriptors and the corresponding component of the preset threshold vector according to each component of the S 3D map descriptors and the corresponding component of the preset threshold vector.
  • the server performs binarization processing on the above size relationship to obtain binarized data of each of the S 3D map descriptors.
  • the server quantifies the absolute value of the difference between each component of the S 3D map descriptors and the corresponding component of the preset threshold vector, to obtain quantized data of each of the S 3D map descriptors.
  • each component included in the preset threshold vector is an arbitrary value.
  • the server encapsulates the binarized data and quantized data of each of the S 3D map descriptors to obtain a code stream of the 3D map.
  • the server sends the code stream of the 3D map to the electronic device.
  • the electronic device acquires the retrieval descriptor.
  • the electronic device may extract the retrieval descriptor from the visual information collected by the sensor.
  • the electronic device decapsulates the code stream of the 3D map to obtain the respective binarized data and quantized data of the S 3D map descriptors, and extract the respective binarized data of the S 3D map descriptors.
  • the electronic device can decode the code stream of the 3D map Encapsulate to obtain the binarized data and quantized data of each of the five 3D map descriptors as shown in FIG. 9 .
  • the electronic device performs a first-level search in the respective binary data of the S 3D map descriptors according to the binarized data of the search descriptors, so as to obtain P 3D map descriptors.
  • the binarized data of the retrieval descriptor is obtained in the following manner: the electronic device can determine the relationship between each component of the retrieval descriptor and the corresponding component of the preset threshold vector according to each component of the retrieval descriptor and the corresponding component of the preset threshold vector. size relationship. The size relationship is binarized to obtain the binarized data of the retrieval descriptor.
  • a first-distance-based retrieval method may be used to perform the first-level retrieval in the binarized data of each of the S 3D map descriptors.
  • the first-level retrieval is performed in five 3D map descriptors (T 1 , T 2 , T 3 , T 4 and T 5 ).
  • the Hamming distance between the binarized data of five 3D map descriptors (T 1 , T 2 , T 3 , T 4 and T 5 ) and the binarized data of the retrieval descriptor can be calculated respectively, according to Hamming distance filters out 3D map descriptors.
  • it is taken as an example to filter out two 3D map descriptors, that is, to filter out T 1 and T 2 shown in the dotted line box in FIG. 9 . That is, P 2.
  • the electronic device decompresses the respective quantized data of the P 3D map descriptors to obtain the respective reconstruction data of the P 3D map descriptors.
  • the electronic device may perform dequantization processing on respective quantized data of the P 3D map descriptors to obtain respective dequantized data of the P 3D map descriptors. According to the respective inverse quantization data and binarized data of the P 3D map descriptors, respective reconstruction data of the P 3D map descriptors are obtained.
  • the electronic device performs a second-level search in the reconstructed data of the P 3D map descriptors, so as to obtain the Q 3D map descriptors.
  • a second distance-based retrieval method can be collected for the second-level retrieval in the respective reconstructed data of the P 3D map descriptors.
  • the second-level retrieval is performed in T1 and T2 according to all components of the retrieval descriptor.
  • the Euclidean distance between the reconstructed data of T 1 and T 2 and the retrieval descriptor can be calculated respectively, and the 3D map descriptor can be screened out according to the Euclidean distance.
  • it is taken as an example to filter out one 3D map descriptor, that is, to filter out T 1 as shown in FIG. 9 . That is, Q 1.
  • the server compresses the S 3D map descriptors of the 3D map to obtain the binarized data and quantized data of each of the S 3D map descriptors, so as to reduce the resource overhead required for transmitting the 3D map.
  • the method for retrieving a 3D map according to the embodiment of the present application is described in detail above with reference to the accompanying drawings, and the apparatus for retrieving the 3D map according to the embodiment of the present application will be introduced below with reference to FIG. 10 and FIG. 11 . It should be understood that the apparatus for retrieving a 3D map can execute the method for retrieving a 3D map in the embodiment of the present application. In order to avoid unnecessary repetition, repeated descriptions are appropriately omitted when introducing the 3D map retrieval apparatus according to the embodiment of the present application below.
  • FIG. 10 is a schematic structural diagram of a 3D map retrieval device provided by an embodiment of the present application.
  • the 3D map retrieval apparatus 1000 may include: a retrieval module 1001 , an extraction module 1002 and a decompression module 1003 .
  • the extraction module 1002 is configured to extract binarized data of S 3D map descriptors from compressed data of S 3D map descriptors, and the S 3D map descriptors correspond to 3D A plurality of 3D map points in the map; a retrieval module 1001, configured to perform i-level retrieval in the binarized data of the S 3D map descriptors according to the binarized data of the retrieved descriptors, so as to obtain P 3D map descriptor; the retrieval descriptor is a feature corresponding to the real environment extracted from the visual information collected by the sensor of the electronic device; the decompression module 1003 is used to perform compression on the compressed data of the P 3D map descriptors decompressing at the mth level to obtain the reconstructed data of the P 3D map descriptors; the decompression process of the compressed data of the P 3D map descriptors at least includes the mth level decompression; the retrieval module 1001 , for performing
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors Matching, N is a positive integer, 0 ⁇ N ⁇ Q, the device also includes: a pose determination module; the pose determination module performs positioning according to the 3D map points corresponding to the N 3D map descriptors to obtain The pose information of the electronic device.
  • the retrieval method adopted by the i-th level retrieval is a retrieval method based on the first distance
  • the retrieval method adopted by the j-th level retrieval is a retrieval method based on the second distance.
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor or, receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor; or, In response to the operation of collecting visual information input by the user, the sensor is triggered to collect visual information from the real environment to obtain the visual information, and the retrieval descriptor is extracted from the visual information, and the retrieval descriptor is Perform binarization processing to obtain binarized data of the retrieval descriptor.
  • the decompression module 1003 is further configured to: perform k-level decompression on the compressed data of the Q 3D map descriptors to obtain Q 3D map descriptions Reconstructed data of the child; according to part or all of the data of the search descriptor, perform r-th level retrieval in the reconstructed data of the Q 3D map descriptors to obtain the N 3D map descriptors;
  • N ⁇ Q ⁇ P the decompression process of the compressed data of the P 3D map descriptors includes the m-level decompression and the k-level decompression, r and k are positive integers, m ⁇ k , j ⁇ r ⁇ L.
  • the compressed data of the P 3D map descriptors includes respective binarization data and quantized data of the P 3D map descriptors
  • the decompression module 1003 is specifically configured to: process the P 3D maps Dequantize the respective quantized data of the descriptors to obtain P dequantized data, and the P dequantized data are used as reconstruction data of the P 3D map descriptors; each of the Q 3D map descriptors Quantized data of the quantized data is dequantized, and Q dequantized data are obtained; according to the Q dequantized data and the binarized data of the Q 3D map descriptors, the Q 3D map descriptors are respectively obtained. of reconstructed data.
  • the compressed data of the P 3D map descriptors includes the respective binarized data and quantized data of the P 3D map descriptors, and the decompression module 1003, specifically It is used for: performing dequantization processing on the respective quantized data of the P 3D map descriptors to obtain P dequantized data; according to the respective binary values of the P dequantized data and the P 3D map descriptors to obtain the respective reconstruction data of the P 3D map descriptors.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data set of the at least one data set Including at least one 3D map descriptor, the retrieval module 1001 is used for: according to the binarization data of the retrieval descriptor, perform i-level retrieval in the S binarized data representing 3D map descriptors, to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the extraction module 1002 is configured to extract the first binarized data of the S 3D map descriptors from the compressed data of the S 3D map descriptors, and the S 3D map descriptors
  • the descriptor corresponds to a plurality of 3D map points in the 3D map
  • the retrieval module 1001 is configured to, according to the first binarized data of the retrieved descriptor, perform the second binarized data of the S 3D map descriptors.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors Matching, N is a positive integer, 0 ⁇ N ⁇ Q, the device further includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors, to obtain the pose information of the electronic device.
  • the i-th level retrieval and the j-th level retrieval adopt a retrieval method based on the first distance
  • the P 3D map descriptors belong to the S 3D map descriptors
  • the position of the first binarization data of each 3D map descriptor of the P 3D map descriptors in the compressed data of the 3D map descriptor is different from the second binarization of the 3D map descriptor The position of the data in the compressed data of the 3D map descriptor, where P ⁇ S.
  • the length of the first binarized data of each 3D map descriptor of the P 3D map descriptors is smaller than the second binary value of each 3D map descriptor of the P 3D map descriptors the length of the optimized data.
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a first binary value of the retrieval descriptor Or, receive the visual information, extract the retrieval descriptor from the visual information, and perform binarization processing on the retrieval descriptor to obtain the retrieval description The first binarized data and the second binarized data; or, in response to the operation of collecting visual information input by the user, triggering the sensor to collect visual information on the real environment to obtain the visual information, from The retrieval descriptor is extracted from the visual information, and the retrieval descriptor is binarized to obtain first binarized data and second binarized data of the retrieval descriptor.
  • an acquisition module configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain a first binary value of the retrieval descriptor
  • receive the visual information extract the retrieval descriptor from the visual information, and perform binarization processing on the retrieval descriptor to
  • the length of the first binarized data of the retrieval descriptor is equal to the length of the first binarized data of each of the S 3D map descriptors, and/or, The length of the second binarized data of the retrieval descriptor is equal to the length of the second binarized data of each 3D map descriptor of the S 3D map descriptors.
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data set of the at least one data set It includes at least one 3D map descriptor, and the retrieval module 1001 is specifically configured to: according to the first binarized data of the retrieved descriptor, perform Searching at the i-level to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the decompression module 1003 is configured to decompress the compressed data of the S 3D map descriptors at the mth level to obtain the first reconstructed data of the S 3D map descriptors, so The S 3D map descriptors correspond to a plurality of 3D map points in the 3D map; the retrieval module 1001 is configured to reconstruct data in the first reconstructed data of the S 3D map descriptors according to part or all of the data of the retrieval descriptors
  • the i-th level search is performed; the search descriptor is extracted from the visual information collected by the sensor of the electronic device and corresponds to the feature of the real environment; the decompression module 1003 also uses performing k-level decompression on the compressed data of the P 3D map descriptors to obtain the second reconstructed data of the P 3D map descriptors; decompressing the compressed data of the S 3D map descriptors
  • the process includes the
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors Matching, N is a positive integer, 0 ⁇ N ⁇ Q, the device further includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors, to obtain the pose information of the electronic device.
  • the i-th level retrieval and the j-th level retrieval adopt a retrieval method based on the second distance
  • the P 3D map descriptors belong to the S 3D map descriptions
  • the distortion degree of the first reconstruction data of each 3D map descriptor of the P 3D map descriptors, and the distortion degree of the second reconstruction data of each 3D map descriptor of the P 3D map descriptors Differently, the distortion degree of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors is used to represent the difference between the first reconstructed data of each 3D map descriptor and the corresponding original 3D map descriptor
  • the degree of difference between the P 3D map descriptors, the distortion degree of the second reconstruction data of each 3D map descriptor is used to represent the second reconstruction data of each 3D map descriptor and the corresponding original 3D map
  • the degree of distortion of the first reconstructed data of each 3D map descriptor of the P 3D map descriptors is greater than the second reconstruction of each 3D map descriptor of the P 3D map descriptors Data distortion.
  • the device further includes: an obtaining module, configured to: receive the retrieval descriptor, and obtain part or all of the data of the retrieval descriptor; or, receive the visual information, and obtain from the extracting the retrieval descriptor from the visual information, and obtaining part or all of the data of the retrieval descriptor; or, in response to the operation of visual information collection input by the user, triggering the sensor to collect visual information from the real environment , to obtain the visual information, extract the retrieval descriptor from the visual information, and acquire part or all of the data of the retrieval descriptor.
  • an obtaining module configured to: receive the retrieval descriptor, and obtain part or all of the data of the retrieval descriptor; or, receive the visual information, and obtain from the extracting the retrieval descriptor from the visual information, and obtaining part or all of the data of the retrieval descriptor; or, in response to the operation of visual information collection input by the user, triggering the sensor to collect visual information from the real environment , to obtain the visual information, extract the
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data set of the at least one data set Including at least one 3D map descriptor, the retrieval module 1001 is specifically configured to: perform the i-th reconstruction data in the m-th reconstructed data of the S representative 3D map descriptors according to part or all of the data of the retrieval descriptor. Level retrieval to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the decompression module 1003 is configured to decompress the compressed data of S 3D map descriptors at the mth level to obtain reconstructed data of S 3D map descriptors, and the S The 3D map descriptor corresponds to multiple 3D map points in the 3D map; the retrieval module 1001 is configured to perform the i-th level in the reconstructed data of the S 3D map descriptors according to part or all of the data of the retrieval descriptor.
  • N 3D map descriptors among the Q 3D map descriptors are used for positioning, and the 3D map points corresponding to the N 3D map descriptors are the 3D map points corresponding to the retrieval descriptors Matching, N is a positive integer, 0 ⁇ N ⁇ Q, the device further includes: a pose determination module; the pose determination module is used to perform positioning according to the 3D map points corresponding to the N 3D map descriptors, to obtain the pose information of the electronic device.
  • the retrieval method adopted by the i-th level retrieval is a retrieval method based on the second distance
  • the retrieval method adopted by the j-th level retrieval is a retrieval method based on the first distance
  • the device further includes: an acquisition module, configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor or, receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor; or, Responding to the operation of collecting visual information input by the user, triggering the sensor to collect visual information on the real environment, obtaining the visual information, extracting the retrieval descriptor from the visual information, and describing the retrieval perform binarization processing to obtain the binarized data of the retrieval descriptor.
  • an acquisition module configured to: receive the retrieval descriptor, and perform binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descriptor or, receiving the visual information, extracting the retrieval descriptor from the visual information, and performing binarization processing on the retrieval descriptor to obtain binarized data of the retrieval descript
  • the S 3D map descriptors are S representative 3D map descriptors, and each of the S representative 3D map descriptors corresponds to at least one data set, and each data set of the at least one data set Including at least one 3D map descriptor, the retrieval module 1001 is specifically configured to: perform the i-th reconstruction data in the m-th reconstructed data of the S representative 3D map descriptors according to part or all of the data of the retrieval descriptor. Level retrieval to obtain at least one representative 3D map descriptor; using the 3D map descriptors in the data sets corresponding to the at least one representative 3D map descriptor as the P 3D map descriptors.
  • the 3D map retrieval device 1000 can execute any one of FIG. 5A to FIG. 5D or the 3D map retrieval device method of the embodiment shown in FIG. 6 or FIG. 7 , or execute S406- Relevant content of S410, or execute relevant content of the embodiment shown in FIG. 9 .
  • FIG. 5A to FIG. 5D the 3D map retrieval device method of the embodiment shown in FIG. 6 or FIG. 7
  • S406- Relevant content of S410 or execute relevant content of the embodiment shown in FIG. 9 .
  • Fig. 11 is a schematic block diagram of an implementation manner of a decoding device 1100 used in the embodiment of the present application.
  • the decoding device 1100 may include a processor 1101 , a memory 1102 and a bus system 1103 .
  • the processor 1101 and the memory 1102 are connected through the bus system 1103, the memory 1102 is used to store instructions, and the processor 1101 is used to execute the instructions stored in the memory 1102 to execute various 3D map retrieval methods described in this application. To avoid repetition, no detailed description is given here.
  • the processor 1101 may be a central processing unit (central processing unit, CPU), and the processor 1101 may also be other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gates or Transistor logic devices, discrete hardware components, and more.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the memory 1102 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1102 .
  • Memory 1102 may include code and data 11021 accessed by processor 1101 using bus 1103 .
  • the memory 1102 may further include an operating system 11023 and an application program 11022, where the application program 11022 includes at least one program that allows the processor 1101 to execute the 3D map retrieval method described in this application.
  • the application program 11022 may include applications 1 to N, which further include a 3D map application that executes the 3D map retrieval method described in this application.
  • bus system 1103 may also include a power bus, a control bus, and a status signal bus.
  • bus system 1103 may also include a power bus, a control bus, and a status signal bus.
  • bus system 1103 may also include a power bus, a control bus, and a status signal bus.
  • bus system 1103 may also include a power bus, a control bus, and a status signal bus.
  • bus system 1103 may also include a power bus, a control bus, and a status signal bus.
  • the decoding apparatus 1100 may further include one or more output devices, such as a display 1104 .
  • display 1104 may be a touch-sensitive display that incorporates a display with a haptic unit operable to sense touch input.
  • the display 1104 may be connected to the processor 1101 via the bus 1103 .
  • the decoding device 1100 can implement the 3D map retrieval method in this application.
  • Computer-readable media may include computer-readable storage media, which correspond to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (eg, according to a communication protocol) .
  • a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave.
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this application.
  • a computer program product may include a computer readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can contain the desired program code in the form of a computer and can be accessed by a computer.
  • any connection is properly termed a computer-readable medium.
  • coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave
  • coaxial cable Wire, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media.
  • Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD) and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce optically with lasers data. Combinations of the above should also be included within the scope of computer-readable media.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • the techniques of the present application may be implemented in a wide variety of devices or devices, including wireless handsets, an integrated circuit (IC), or a group of ICs (eg, a chipset).
  • IC integrated circuit
  • a group of ICs eg, a chipset
  • Various components, modules, or units are described in this application to emphasize functional aspects of means for performing the disclosed techniques, but do not necessarily require realization by different hardware units. Indeed, as described above, the various units may be combined in a codec hardware unit in conjunction with suitable software and/or firmware, or by interoperating hardware units (comprising one or more processors as described above) to supply.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Processing Or Creating Images (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)

Abstract

一种3D地图的检索方法和装置，有益于提升检索性能，属于3D地图技术领域。所述3D地图的检索方法包括：获取S个3D地图描述子的二值化数据，根据检索描述子的二值化数据（S101A），在S个3D地图描述子的二值化数据中进行第i级检索，筛选得到P个3D地图描述子（S102A），对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的重构数据（S103A），根据检索描述子的部分数据或全部数据，在P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子（S104A）。可以提升检索速度，并可以有益于保证检索准确率。

Description

3D地图的检索方法和装置 技术领域
本申请涉及定位技术，尤其涉及一种3D地图的检索方法和装置。
背景技术
虚拟现实(Virtual Reality，VR)、增强现实(Augmented Reality，AR)和混合现实(Mixed Reality，MR)技术是近年来新兴的多媒体虚拟场景技术。这一类技术可以创建虚拟现实，并将虚拟现实和真实世界叠加，产生新的可视化环境和交互体验。在这类应用中，电子设备需要确定自身在当前环境中的位姿信息，以精确的实现虚拟物体与真实场景的融合。
另一方面，在自动驾驶、自主导航、无人机自动巡检、工业机器人等应用中，车辆、无人机、机器人等载具，需要通过确定其上搭载的电子设备的位姿来确定自身在当前环境中的位姿，以进行精确的路径规划、导航、检测、操控。
针对以上应用中，需确定电子设备在当前环境中的位姿的问题，一个典型的解决方案是：电子设备从服务器或者其他设备接收所处环境的三维(3-dimension，3D)地图，再通过本地传感器采集所处环境中的视觉信息，根据采集的视觉信息在下载的3D地图中进行检索，以确定电子设备的当前位姿。
然而，原始的3D地图通常包含的数据量巨大，检索的计算量通常极大，需要消耗大量计算资源，且耗时较长，影响用户体验。
发明内容
本申请实施例提供了一种3D地图的检索方法和装置，可以提升检索性能。
第一方面，本申请实施例提供一种3D地图的检索方法，该方法可以包括：从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，S个3D地图描述子对应3D地图中的多个3D地图点。根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子。检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的重构数据。P个3D地图描述子的压缩数据的解压缩过程(完全解压缩过程)至少包括第m级解压缩。根据检索描述子的部分数据或全部数据，在P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
可见，本实施例中，根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，筛选得到P个3D地图描述子，对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的重构数据，根据检索描述子的部分数据或全部数据， 在P个3D地图描述子的重构数据中进行第j级检索，以筛选出更少个数的3D地图描述子。至少两级检索中的一级检索使用压缩数据中的二值化数据，可以提升检索速度，另一级检索使用解压缩得到的重构数据，可以保证检索准确性。相较于在完全解压得到的3D地图的重构数据中进行检索，本申请实施例提供的3D地图的检索方法可以提升检索性能。换言之本申请实施例提供的3D地图的检索方法可以提升检索性能。
第j级可以是多级检索中的中间级，即j<L，或者，也可以是多级检索中的最后一级，即j＝L。
一种示例，上述第m级解压缩可以是阶段性解压缩，可以理解为完全解压缩的子过程，或理解为部分解压缩。另一种示例，上述第m级解压缩可以是完全解压缩。
一种可能的设计中，Q个3D地图描述子中的N个3D地图描述子用于定位，N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，上述方法还可以包括：根据N个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿信息。
本实施例中，当检索得到的N个3D地图描述子对应的3D地图点与检索描述子对应的地图点匹配时，可以根据N个3D地图描述子对应的3D地图点进行定位，有益于提升定位性能。
一种可能的设计中，第i级检索所采用的检索方式为基于第一距离的检索方式，第j级检索所采用的检索方式为基于第二距离的检索方式。
该第一距离可以包括汉明距离等利用二值化数据得到的距离。例如，第一距离可以是汉明距离。又例如，第一距离可以是对两个二值化数据的对应位取差值的绝对值，并将所有差值的绝对值相加得到的一种距离。基于第一距离的检索指的是通过计算第一距离，确定相关度或相似度，以筛选3D地图描述子。第二距离可以包括但不限于欧式距离、内积距离、余弦距离、曼哈顿距离等。基于第二距离的检索指的是通过计算第二距离，确定相关度或相似度，以筛选3D地图描述子。
本实施例中，基于第一距离进行检索，可以提升检索速度，基于第二距离进行检索，可以提升检索的准确性。
一种可能的设计中，该方法还可以包括：接收检索描述子，并对检索描述子进行二值化处理，以得到检索描述子的二值化数据。用于执行第一方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器(例如云服务器))可以接收其他设备发送的检索描述子。例如，用于执行第一方面所述的检索方法的执行主体接收其他电子设备采集和提取的检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
一种可能的设计中，该方法还可以包括：接收所述视觉信息，从视觉信息中提取检索描述子，并对检索描述子进行二值化处理，以得到检索描述子的二值化数据；用于执行第一方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以接收其他设备发送的视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
一种可能的设计中，该方法还可以包括：响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中 提取所述检索描述子，对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。用于执行第一方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以采集得到所述视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
本申请实施例的3D地图的检索方法，可以在3D地图中对不同应用场景中的检索描述子进行检索，具有广泛的适用性。
一种可能的设计中，在N<Q的情况下，该方法还包括：对Q个3D地图描述子的压缩数据进行第k级解压缩，以得到Q个3D地图描述子的重构数据。根据检索描述子的部分数据或全部数据，在Q个3D地图描述子的重构数据中进行第r级检索，以得到N个3D地图描述子。其中，N<Q<P，P个3D地图描述子的压缩数据的解压缩过程包括第m级解压缩(阶段性解压缩)和第k级解压缩(阶段性解压缩)，r和k为正整数，m<k，j<r≤L。
本实施例，根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，筛选得到P个3D地图描述子，对P个3D地图描述子的压缩数据进行第m级解压缩(阶段性解压缩)，得到P个3D地图描述子的重构数据，根据检索描述子的部分数据或全部数据，在P个3D地图描述子的重构数据中进行第j级检索，以筛选出Q个3D地图描述子。之后，对Q个3D地图描述子的压缩数据进行第k级解压缩(阶段性解压缩)，以得到Q个3D地图描述子的重构数据，根据检索描述子的全部数据，在Q个3D地图描述子的重构数据中进行第r级检索，以得到N个3D地图描述子，该N个3D地图描述子可以用于定位。至少三级检索中的一级检索使用压缩数据中的二值化数据，可以提升检索速度，另两级检索使用不同解压缩得到的重构数据，可以在提升检索速度的同时，保证检索准确性。
具体的，第m级解压缩和第k级解压缩各自得到的重构数据的解压缩程度不同或失真度不同。
一种可能的设计中，P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，对P个3D地图描述子的压缩数据进行第m级解压缩得到P个3D地图描述子的重构数据，包括：对P个3D地图描述子各自的量化数据进行反量化处理，得到P个反量化数据，P个反量化数据用作P个3D地图描述子的重构数据。对Q个3D地图描述子的压缩数据进行第k级解压缩得到Q个3D地图描述子的重构数据，包括：对Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据，根据Q个反量化数据和Q个3D地图描述子各自的二值化数据，得到Q个3D地图描述子各自的重构数据。
本实施例，通过对P个3D地图描述子各自的量化数据进行反量化处理，得到P个3D地图描述子各自的反量化数据，基于P个3D地图描述子各自的反量化数据(用作本级检索的重构数据)进行第j级检索，筛选得到Q个3D地图描述子，通过对Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据，根据Q个反量化数据和Q个3D地图描述子各自的二值化数据，得到Q个3D地图描述子各自的重构数据，基于Q个3D地图描述子各自的重构数据进行第r级检索，以得到N个3D地图描述子。第j级检索的重构数据的失真度大于第r级检索的重构数据的失真度，即先粗糙检索，提升检索速度，后精细检索，提升检索的准确性。
一种可能的设计中，在N＝Q的情况下，P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，对P个3D地图描述子的压缩数据进行第m级解压缩得 到P个3D地图描述子的重构数据，包括：对P个3D地图描述子各自的量化数据，进行反量化处理，得到P个反量化数据，根据P个反量化数据和P个3D地图描述子各自的二值化数据，得到P个3D地图描述子各自的重构数据。
本实施例，第j级检索可以是多级检索的最后一级，最后一级基于完全解压缩得到重构数据进行检索，可以提升检索的准确性。
一种可能的设计中，S个3D地图描述子为S个代表3D地图描述子，S个代表3D地图描述子各自分别对应至少一个数据集合，至少一个数据集合的各个数据集合包括至少一个3D地图描述子，根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子，包括：根据检索描述子的二值化数据，在S个代表3D地图描述子的二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子，将至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为P个3D地图描述子。
每个数据集合中的3D地图描述子具有一定相关性或相似性。代表3D地图描述子用于表征对应的数据集合中的3D地图描述子的共性或相似特征。其中，P可以大于S。P<T，S<T。
本实施例，通过在较少数量的代表3D地图描述子进行一级检索，以提升检索速度，之后可以在代表3D地图描述子对应的数据集合中进行一级或多级检索，以提升检索准确率。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
一种可能的设计中，Q＜P＜S＜T。即多级检索中3D地图描述子的数量逐级减少。
一种可能的设计中，该方法还可以包括：根据检索描述子的各个分量和预设阈值向量的对应分量，确定检索描述子的各个分量与预设阈值向量的对应分量之间的大小关系。对大小关系进行二值化处理，得到检索描述子的二值化数据。
一种可能的设计中，根据检索描述子的各个分量和预设阈值向量的对应分量，确定检索描述子的各个分量与预设阈值向量的对应分量之间的大小关系，包括：分别将检索描述子的各个分量与预设阈值向量的对应分量相减，得到各个分量的差值；根据各个分量的差值，确定所述各个分量符号。对大小关系进行二值化处理，得到检索描述子的二值化数据，包括：对各个分量符号进行二值化处理，得到检索描述子的二值化数据。
第二方面，本申请实施例提供一种3D地图的检索方法，该方法可以包括：从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第一二值化数据，S个3D地图描述子对应3D地图中的多个3D地图点。根据检索描述子的第一二值化数据，在S个3D地图描述子的第一二值化数据中进行第i级检索，以得到P个3D地图描述子。检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。从P个3D地图描述子的压缩数据中提取P个3D地图描述子的第二二值化数据。根据检索描述子的第二二值化数据，在P个3D地图描述子的第二二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
可见，本实施例中，从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第一二值化数据，根据检索描述子的第一二值化数据，在S个3D地图描述子的第一二值化数据 中进行第i级检索，筛选得到P个3D地图描述子。从P个3D地图描述子的压缩数据中提取P个3D地图描述子的第二二值化数据，根据检索描述子的第二二值化数据，在P个3D地图描述子的第二二值化数据中进行第j级检索，以筛选出更少个数的3D地图描述子。通过从压缩数据中提取一级检索的3D地图描述子的二值化数据，与一级检索交替进行，以在压缩状态的3D地图中进行至少两级检索，得到检索后的3D地图描述子。任意两级检索各自所使用的3D地图描述子的二值化数据所携带的信息内容不同或各自所使用的3D地图描述子的二值化数据的信息完整程度不同。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
第一二值化数据和第二二值化数据可以多级检索中任意两级中所使用的二值化数据，其二值化处理方式可以不同，或长度不同。
一种可能的设计中，Q个3D地图描述子中的N个3D地图描述子用于定位，N个3D地图描述子对应的3D地图点与检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，上述方法还包括：根据N个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿信息。
一种可能的设计中，第i级检索和第j级检索所采用的检索方式均为基于第一距离的检索，P个3D地图描述子属于S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据在所述3D地图描述子的压缩数据中的位置，不同于所述3D地图描述子的第二二值化数据在所述3D地图描述子的压缩数据中的位置，其中，P＜S。例如，这个位置可以是起始位置，也可以是结束位置等。
可见，通过差异化的至少两级检索，以实现检索速度和检索准确率的综合提升。
一种可能的设计中，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，小于所述P个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
可见，第i级检索中的各个3D地图描述子的第一二值化数据的长度较短，可以提升检索速度，第j级检索中的各个3D地图描述子的第二二值化数据的长度较长，可以提升检索准确率。通过差异化的至少两级检索，以实现检索速度和检索准确率的综合提升。
一种可能的设计中，上述方法还包括：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据。用于执行第二方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器(例如云服务器))可以接收其他设备发送的检索描述子。例如，用于执行第二方面所述的检索方法的执行主体接收其他电子设备采集和提取的检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的第一二值化数据和第二二值化数据。
一种可能的设计中，上述方法还包括：接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据。用于执行第二方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以接收其他设备发送的视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的第一二值化数据和第二二值化数据。
一种可能的设计中，上述方法还包括：响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提 取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据。用于执行第二方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以采集得到所述视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的第一二值化数据和第二二值化数据。
一种可能的设计中，所述检索描述子的第一二值化数据的长度等于所述S个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，和/或，所述检索描述子的第二二值化数据的长度等于述S个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
本实施例，通过计算等长度的二值化数据的第一距离，以在3D地图中进行检索，可以提升检索速度。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述根据检索描述子的第一二值化数据，在所述S个3D地图描述子的第一二值化数据中进行第i级检索，以得到P个3D地图描述子，包括：根据所述检索描述子的第一二值化数据，在所述S个代表3D地图描述子的第一二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
一种可能的设计中，Q＜P＜S＜T。即多级检索中3D地图描述子的数量逐级减少。
第三方面，本申请实施例提供一种3D地图的检索方法，该方法可以包括：对S个3D地图描述子的压缩数据进行第m级解压缩，以得到S个3D地图描述子的第一重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点。根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的第一重构数据中进行第i级检索，以得到P个3D地图描述子。所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。对所述P个3D地图描述子的压缩数据进行第k级解压缩，得到所述P个3D地图描述子的第二重构数据，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，或者，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第k级解压缩，所述第k级解压缩包括第m级解压缩。根据所述检索描述子的部分数据或全部数据，在所述P个3D地图描述子的第二重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j、k和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
可见，本实施例中，对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的第一重构数据，根据检索描述子的部分数据或全部数据，在S个3D地图描述子的第一重构数据中进行第i级检索，筛选出P个3D地图描述子。对P个3D地图描述子的压缩数据进行第k级解压缩，得到P个3D地图描述子的第二重构数据，根据检索描述子的部分数据或全部数据，在P个3D地图描述子的第二重构数据中进行第j级检索，以筛选出更少个数的3D地图描述子。通过解压缩得到一级检索的3D地图描述子的一种重构数据，与一级检索交替进行。至少两级检索中存在一级检索为阶段性解压缩，可以提升检索速度，至少两级检索的解压缩差异化设置，可以保证检索准确率。任意两级检索各自所 使用的3D地图描述子的一种重构数据的解压缩程度不同或失真度不同。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
一种可能的设计中，Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，上述方法还包括：根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
本实施例中，当检索得到的N个3D地图描述子对应的3D地图点与检索描述子对应的地图点匹配时，可以根据N个3D地图描述子对应的3D地图点进行定位，有益于提升定位性能。
一种可能的设计中，所述第i级检索和所述第j级检索所采用的检索方式均为基于第二距离的检索方式，所述P个3D地图描述子属于所述S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，与所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度不同，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度用于表示所述各个3D地图描述子的第一重构数据与相应的原始3D地图描述子之间的差别程度，所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度用于表示所述各个3D地图描述子的第二重构数据与相应的原始3D地图描述子之间的差别程度，其中，P＜S。
可见，通过差异化的至少两级检索，以实现检索速度和检索准确率的综合提升。
一种可能的设计中，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，大于所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度。
可见，第i级检索中的各个3D地图描述子的第一重构数据的的失真度可以大于P个3D地图描述子中各个3D地图描述子的第二重构数据的失真度。通过前后两级检索中前一级检索的失真度大于后一级检索的失真度，以在前一级检索中进行快速检索，提升检索速度，在后一级检索中进行精细检索，提升检索准确率。
一种可能的设计中，上述方法还包括：接收所述检索描述子，并获取所述检索描述子的部分数据或全部数据。用于执行第三方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器(例如云服务器))可以接收其他设备发送的检索描述子。例如，用于执行第三方面所述的检索方法的执行主体接收其他电子设备采集和提取的检索描述子。之后根据检索需求获取所述检索描述子的部分数据或全部数据。
一种可能的设计中，上述方法还包括：接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据。用于执行第三方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以接收其他设备发送的视觉信息，并从中提取检索描述子。之后根据检索需求获取所述检索描述子的部分数据或全部数据。
一种可能的设计中，上述方法还包括：响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据。用于执行第三方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以采集得到所述视 觉信息，并从中提取检索描述子。之后根据检索需求获取所述检索描述子的部分数据或全部数据。
本申请实施例的3D地图的检索方法，可以在3D地图中对不同应用场景中的检索描述子进行检索，具有广泛的适用性。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的第一重构数据中进行第i级检索，以得到P个3D地图描述子，包括：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的第一重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
每个数据集合中的3D地图描述子具有一定相关性或相似性。代表3D地图描述子用于表征对应的数据集合中的3D地图描述子的共性或相似特征。其中，P可以大于S。P<T，S<T。
本实施例，通过在较少数量的代表3D地图描述子进行一级检索，以提升检索速度，之后可以在代表3D地图描述子对应的数据集合中进行一级或多级检索，以提升检索准确率。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
一种可能的设计中，Q＜P＜S＜T。即多级检索中3D地图描述子的数量逐级减少。
第四方面，本申请实施例提供一种3D地图的检索方法，该方法可以包括：对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点。根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的重构数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。从所述P个3D地图描述子的压缩数据中提取所述P个描述子的二值化数据。根据所述检索描述子的二值化数据，在所述P个3D地图描述子的二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、m和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
可见，本实施例中，至少两级检索中存在一级检索为解压缩，可以提升检索准确性，至少两级检索中存在一级检索为从压缩数据中提取，可以提升检索速度。相较于在完全解压得到的3D地图的重构数据中进行检索，本申请实施例提供的3D地图的检索方法可以提升检索速度，并可以保证检索准确率。
一种可能的设计中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述方法还包括：根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
一种可能的设计中，所述第i级检索所采用的检索方式为基于第二距离的检索方式，所述第j级检索所采用的检索方式为基于第一距离的检索方式。
本实施例，基于第二距离的检索方式可以提升检索准确性，基于第一距离的检索方式可以提升检索速度，从而实现检索准确性和检索速度的综合提升。
一种可能的设计中，上述方法还包括：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；用于执行第四方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器(例如云服务器))可以接收其他设备发送的检索描述子。例如，用于执行第一方面所述的检索方法的执行主体接收其他电子设备采集和提取的检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
一种可能的设计中，上述方法还包括：接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；用于执行第四方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以接收其他设备发送的视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
一种可能的设计中，上述方法还包括：响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。用于执行第四方面所述的检索方法的执行主体(可以是电子设备，也可以是服务器)可以接收其他设备发送的视觉信息，并从中提取检索描述子。之后对检索描述子进行二值化处理，以得到检索描述子的二值化数据。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的重构数据中进行第i级检索，以得到P个3D地图描述子，包括：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
每个数据集合中的3D地图描述子具有一定相关性或相似性。代表3D地图描述子用于表征对应的数据集合中的3D地图描述子的共性或相似特征。其中，P可以大于S。P<T，S<T。
本实施例，通过在较少数量的代表3D地图描述子进行一级检索，以提升检索速度，之后可以在代表3D地图描述子对应的数据集合中进行一级或多级检索，以提升检索准确率。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
一种可能的设计中，Q＜P＜S＜T。即多级检索中3D地图描述子的数量逐级减少。
结合第一方面或第一方面的任意一种可能的设计，或者，结合第二方面或第二方面的任意一种可能的设计，或者，结合第三方面或第三方面的任意一种可能的设计，或者，结合第四方面或第四方面的任意一种可能的设计，还存在如下可能的设计：
一种可能的设计中，检索描述子包括区域描述子或3D地图点描述子。相应的，S个3D地图描述子可以包括S个区域描述子或S个3D地图点描述子。
第五方面，本申请实施例提供一种3D地图的检索装置，该装置可以为电子设备或者服务器中的芯片或者片上系统，还可以为电子设备或者服务器中用于实现第一方面或第一方面的任一可能的实施方式所述的方法的功能模块。举例来说，该3D地图的检索装置，包括：检索模块、提取模块和解压缩模块。
一种可能的设计中，提取模块，用于从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块，用于根据检索描述子的二值化数据，在所述S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；解压缩模块，用于对所述P个3D地图描述子的压缩数据进行第m级解压缩，得到所述P个3D地图描述子的重构数据；所述P个3D地图描述子的压缩数据的解压缩过程至少包括所述第m级解压缩；所述检索模块，用于根据所述检索描述子的部分数据或全部数据，在所述P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
一种可能的设计中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
一种可能的设计中，所述第i级检索所采用的检索方式为基于第一距离的检索方式，所述第j级检索所采用的检索方式为基于第二距离的检索方式。
一种可能的设计中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
一种可能的设计中，在N<Q的情况下，所述解压缩模块还用于：对所述Q个3D地图描述子的压缩数据进行第k级解压缩，以得到Q个3D地图描述子的重构数据；根据所述检索描述子的部分数据或全部数据，在所述Q个3D地图描述子的重构数据中进行第r级检索，以得到所述N个3D地图描述子；其中，N<Q<P，所述P个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，r和k为正整数，m<k，j<r≤L。
一种可能的设计中，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压模块具体用于：对所述P个3D地图描述子各自的量化数据进行反量化处理，得到P个反量化数据，所述P个反量化数据用作所述P个3D地图描述子的重构数据；对所述Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据；根据所述Q个反量化数据和所述Q个3D地图描述子各自的二值化数据，得到所述Q个3D地图描述子各自的重构数据。
一种可能的设计中，在N＝Q的情况下，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压缩模块，具体用于：对所述P个3D地图描述子各自的量化数据，进行反量化处理，得到P个反量化数据；根据所述P个反量化数据和所述P个3D地图描述子各自的二值化数据，得到所述P个3D地图描述子各自的重构数据。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块用于：根据所述检索描述子的二值化数据，在所述S个代表3D地图描述子的二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
第六方面，本申请实施例提供一种3D地图的检索装置，该装置可以为电子设备或者服务器中的芯片或者片上系统，还可以为电子设备或者服务器中用于实现第一方面或第一方面的任一可能的实施方式所述的方法的功能模块。举例来说，该3D地图的检索装置，包括：检索模块、提取模块和解压缩模块。
一种可能的设计中，提取模块，用于从S个3D地图描述子的压缩数据中提取所述S个3D地图描述子的第一二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块，用于根据检索描述子的第一二值化数据，在所述S个3D地图描述子的第一二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；所述提取模块，还用于从所述P个3D地图描述子的压缩数据中提取所述P个3D地图描述子的第二二值化数据；所述检索模块，用于根据所述检索描述子的第二二值化数据，在所述P个3D地图描述子的第二二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
一种可能的设计中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
一种可能的设计中，所述第i级检索和所述第j级检索所采用的检索方式均为基于第一距离的检索，所述P个3D地图描述子属于所述S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据在所述3D地图描述子的压缩数据中的位置，不同于所述3D地图描述子的第二二值化数据在所述3D地图描述子的压缩数据中的位置，其中，P＜S。
一种可能的设计中，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，小于所述P个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
一种可能的设计中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据；或 者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据。
一种可能的设计中，所述检索描述子的第一二值化数据的长度等于所述S个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，和/或，所述检索描述子的第二二值化数据的长度等于述S个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块具体用于：根据所述检索描述子的第一二值化数据，在所述S个代表3D地图描述子的第一二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
第七方面，本申请实施例提供一种3D地图的检索装置，该装置可以为电子设备或者服务器中的芯片或者片上系统，还可以为电子设备或者服务器中用于实现第一方面或第一方面的任一可能的实施方式所述的方法的功能模块。举例来说，该3D地图的检索装置，包括：检索模块、提取模块和解压缩模块。
一种可能的设计中，解压缩模块，用于对S个3D地图描述子的压缩数据进行第m级解压缩，以得到S个3D地图描述子的第一重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块，用于根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的第一重构数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；所述解压缩模块，还用于对所述P个3D地图描述子的压缩数据进行第k级解压缩，得到所述P个3D地图描述子的第二重构数据，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，或者，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第k级解压缩，所述第k级解压缩包括第m级解压缩；检索模块，用于根据所述检索描述子的部分或全部，在所述P个3D地图描述子的第二重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j、k和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
一种可能的设计中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
一种可能的设计中，所述第i级检索和所述第j级检索所采用的检索方式均为基于第二距离的检索方式，所述P个3D地图描述子属于所述S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，与所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度不同，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度用于表示所述各个3D地图描述子的第一重构数据与相应的原始3D地图描述子之间的差别程度，所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失 真度用于表示所述各个3D地图描述子的第二重构数据与相应的原始3D地图描述子之间的差别程度，其中，P＜S。
一种可能的设计中，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，大于所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度。
一种可能的设计中，所述装置还包括：获取模块，用于：接收所述检索描述子，并获取所述检索描述子的部分数据或全部数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块具体用于：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的第m重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
第八方面，本申请实施例提供一种3D地图的检索装置，该装置可以为电子设备或者服务器中的芯片或者片上系统，还可以为电子设备或者服务器中用于实现第一方面或第一方面的任一可能的实施方式所述的方法的功能模块。举例来说，该3D地图的检索装置，包括：检索模块、提取模块和解压缩模块。
一种可能的设计中，解压缩模块，用于对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块，用于根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的重构数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；提取模块，用于从所述P个3D地图描述子的压缩数据中提取所述P个描述子的二值化数据；所述检索模块，还用于根据所述检索描述子的二值化数据，在所述P个3D地图描述子的二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、m和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
一种可能的设计中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
一种可能的设计中，所述第i级检索所采用的检索方式为基于第二距离的检索方式，所述第j级检索所采用的检索方式为基于第一距离的检索方式。
一种可能的设计中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理， 以得到所述检索描述子的二值化数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
一种可能的设计中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块具体用于：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的第m重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
第九方面，本申请实施例提供一种3D地图的检索装置，包括：一个或多个处理器；存储器，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如第一至第四任一方面或第一至第四任一方面任一项可能的设计所述的方法。
第十方面，本申请实施例提供一种计算机可读存储介质，其特征在于，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行第一至第四任一方面或第一至第四任一方面任一项可能的设计所述的方法。
第十一方面，本申请实施例提供一种计算机程序或计算机程序产品，当计算机程序或计算机程序产品在计算机上被执行时，使得计算机实现如上述第一至第四任一方面及其任一种可能的实施方式所描述的方法。
应当理解的是，本申请实施例的第五至十一方面与本申请实施例的第一至第四方面的技术方案一致，各方面及对应的可行实施方式所取得的有益效果相似，不再赘述。
附图说明
图1为本申请实施例所提供的一种应用架构的示意图；
图2为本申请实施例提供的电子设备20的一种结构示意图；
图3为本申请实施例提供的服务器30的一种结构示意图；
图4a-图4f为本申请实施例提供的各种应用场景的示意图；
图4g为本申请实施例提供的电子设备显示的一种用户界面(例如3D地图应用的用户界面)的示意图；
图5A至图5D是本申请实施例提供的一种3D地图的检索方法的片段流程示意图；
图6是本申请实施例提供的一种3D地图的检索方法的流程示意图；
图7是本申请实施例提供的一种3D地图的检索方法的流程示意图；
图8是本申请实施例提供的一种3D地图的检索方法的流程示意图；
图9是本申请实施例提供的一种3D地图的检索方法的处理过程示意图
图10为本申请实施例提供的一种3D地图的检索装置的结构示意图；
图11为本申请实施例提供的一种译码装置1100的示意性框图。
具体实施方式
下面结合本申请实施例中的附图对本申请实施例进行描述。本申请的实施方式部分使用的术语仅用于对本申请的具体实施例进行解释，而非旨在限定本申请。
本申请的说明书实施例和权利要求书及附图中的术语“第一”、“第二”等仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元。方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请实施例或所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。如本文所使用的，单数形式的“一”、“某”和“该”旨在也包括复数形式，除非上下文另有明确指示。还将理解，术语“包括”、“具有”、“包含”和/或“含有”在本文中使用时指定所陈述的特征、整数、步骤、操作、要素、和/或组件的存在，但并不排除一个或多个其他特征、整数、步骤、操作、要素、组件和/或其群组的存在或添加。
需要说明的是，在本申请实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。
图1为本申请实施例所提供的一种应用架构的示意图，如图1所示，该应用架构包括多个电子设备和服务器，其中，多个电子设备可以包括第一电子设备、一个或多个第二电子设备(图1中以两个第二电子设备为例)，该一个或多个第二电子设备是除第一电子设备外的若干个电子设备。多个电子设备和服务器之间、多个电子设备彼此之间均可进行通信，例如，对于应用架构中的任意一个设备，可以通过无线保真(wireless-fidelity，WiFi)通信、蓝牙通信或者蜂窝2/3/4/5代(2/3/4/5generation，2G/3G/4G/5G)通信等方式与其它设备进行通信，应当理解的，服务器和电子设备之间还可以采用其他通信方式，包括未来的通信方式，对此不做具体限定。需要说明的是，本申请实施例中的“一个或多个第二电子设备”仅用于表示除第一电子设备外的其 它电子设备，但并不限定多个电子设备的类型是否相同。
上述电子设备，可以是配置有摄像头和显示组件的各种类型的设备，例如，电子设备可以是手机、平板电脑、笔记本电脑、录像机等终端设备(图1中以电子设备是手机为例)，电子设备也可以是用于虚拟场景交互的设备，包括VR眼镜、AR设备，MR互动设备等，电子设备还可以是智能手表、智能手环等可穿戴电子设备，电子设备还可以是车辆、无人驾驶车辆、无人机、工业机器人等载具中所搭载的设备。本申请实施例对电子设备的具体形式不做具体限定。
此外，上述电子设备也可以被称为用户装备(user equipment，UE)、订户站、移动单元、订户单元、无线单元、远程单元、移动设备、无线设备、无线通信设备、远程设备、移动订户站、终端设备、接入终端、移动终端、无线终端、智能终端、远程终端、手持机、用户代理、移动客户端、客户端、或其他某个合适的术语。
上述服务器可以是一台或多台物理服务器(图1中以一台物理服务器为例)，也可以是计算机集群，还可以是云计算场景的虚拟机或者云端服务器，等等。
本申请实施例中，电子设备可以安装VR应用、AR应用或者MR应用等虚拟场景应用程序(application，APP)，并可以基于用户的操作(例如点击、触摸、滑动、抖动、声控等)运行VR应用、AR应用或者MR应用。电子设备可以通过传感器采集环境中任意物体的视觉信息，再根据采集的视觉信息在显示组件上显示虚拟物体，该虚拟物体可以是VR场景、AR场景或MR场景中的虚拟物体(即虚拟环境中的物体)。
本申请实施例中，电子设备可以安装导航、检测、控制类应用程序，并基于用户的操控或是预设的程序来运行相应的应用。电子设备可以基于自身在当前环境中的位姿和其他状态信息，进行路径规划、物体检测、载具操控等应用。
位姿即电子设备的位置和朝向信息，可以是在世界坐标系中的绝对位姿，也可以是相对于环境中某个点的相对位姿。
本申请实施例所涉及的视觉信息，包括但不限于摄像头采集得到的图像视频(不带深度信息)、深度感应器(depth sensor)采集得到的带深度信息的图像视频、激光雷达(LiDAR)采集得到的数据、毫米波雷达(RaDAR)采集得到的数据。
需要说明的是，本申请实施例中，电子设备中的虚拟场景应用程序可以是电子设备自身内置的应用程序，也可以是用户自行安装的第三方服务商提供的应用程序，对此不做具体限定。
本申请实施例中，电子设备还可以配置有即时定位与地图构建(simultaneous localization and mapping，SLAM)系统，SLAM系统能够在完全未知的环境中创建地图，并利用该地图进行定位、位姿(位置与姿态)确定、导航等。本申请实施例中将SLAM系统创建的地图称作SLAM地图，SLAM地图可以理解为是SLAM系统根据采集设备采集到的环境信息绘制出的地图，采集设备可以包括电子设备中的视觉信息采集装置和惯性测量单元(inertial measurement unit，IMU)，其中，视觉信息采集装置例如可以包括摄像头、相机、深度相机、激光雷达、毫米波雷达等，IMU例如可以包括陀螺仪、加速度计等传感器。
本申请实施例中将SLAM地图亦称作3D地图。需要说明的是，3D地图包括但 不限于SLAM地图，其还可以包括采用其它技术创建的三维地图，本申请实施例对此不做具体限定。
在一种可能的实现方式中，3D地图可以包括多个3D地图点，相应的，3D地图的数据可以包括多个3D地图点的数据。3D地图点是环境中令人感兴趣的或者具有显著特征的点。
3D地图点的一种可能的获取方式为，使用激光雷达、无人机视角的空中摄影(倾斜摄影)、高清全景相机、高清工业相机等多种设备拍摄，通过ORB、尺度不变特征变换(scale-invariant feature transform，SIFT)、加速版具有鲁棒性特征(speeded up robust features，SURF)、二元鲁棒独立基本特征(binary robust independent elementary features，BRIEF)、二进制鲁棒不变可伸缩关键点(binary robust invariant scalable keypoints，BRISK)、快速视网膜关键点(fast retina keypoint，FREAK)、D2Net或者基于自监督训练的特征点检测和描述符提取方法(SuperPoint)等方法从上述设备拍摄到的数据中提取。
3D地图点的数据可以包括：
(1)3D地图点描述子
3D地图点描述子是一条向量(vector)，用来表示对应3D地图点的局部特征。在视觉定位算法中，3D地图点描述子用于进行3D地图点间的匹配。一种可能的做法是，计算两个3D地图点描述子之间的距离(可以是欧式距离、内积距离、汉明距离等)，当距离小于一个阈值时，视为两个3D地图点匹配。
(2)3D地图点空间位置
3D地图点空间位置可以采用三维空间轴线上的X、Y、Z表示，也可以采用经纬度、海拔高度表示，还可以采用极坐标等方式表示，本申请实施例对3D地图点空间位置的表示方法不做具体限制。3D地图点空间位置可以是3D地图点的绝对位置，也可以是3D地图点的相对位置，例如，以整个区域的中心位置为原点，所有3D地图点空间位置都是相对于原点的空间位置的偏移位置。
本申请实施例中，可以给每个3D地图点分配一个编号写入3D地图的数据中，也可以利用多个3D地图点在内存中的存储顺序隐含表示3D地图点的编号。需要说明的是，3D地图包含的多个3D地图点之间的先后顺序没有实际意义，因此前述编号可以认为是用于标识3D地图点的标识符，以对各个3D地图点加以区分，但该编号并不用于限定多个3D地图点之间的先后顺序，例如，3D地图包含3个3D地图点，其编号分别是1、2和3，对该3个3D地图点进行的处理，可以按照1、2、3的顺序进行，也可以按照3、2、1的顺序进行，还可以按照2、1、3的顺序进行，等等。
在一种可能的实现方式中，3D地图的数据还包括多个区域描述子，该多个区域描述子中的任意一个区域描述子用于描述上述多个3D地图点中的部分或全部3D地图点的特征，即针对多个区域描述子中的任意一个区域描述子，该区域描述子可以用于描述多个3D地图点中的部分或全部3D地图点的特征，这样区域描述子和3D地图点是一对多的关系。而多个3D地图点中的每个3D地图点的特征可以被多个区域描述子中的部分或全部区域描述子描述，这样3D地图点和区域描述子是一对多的关 系。由此可见，多个区域描述子和多个3D地图点之间是多对多的关系。区域描述子的生成方法包括但不限于词袋(bag of words，BOW)、局部聚集描述符向量(vector of locally aggregated descriptors，VLAD)等传统方法以及基于NetVLAD、人工智能(artificial intelligence，AI)的新型方法。同理，多个区域描述子也可以用编号进行标识，以对多个区域描述子进行区分，但是该编号同样不是对多个区域描述子之间的先后顺序的限定。
在一种可能的实现方式中，3D地图的数据还包括3D地图点和描述子的对应关系，在该对应关系中明确的描述了任意一个描述子对应于哪些3D地图点，以及任意一个3D地图点对应于哪些描述子。
可选的，上述对应关系可以显式的用区域描述子的编号和3D地图点的编号之间的对应表进行描述，例如，3D地图包含3个区域描述子，编号是T1～T3，6个3D地图点，其中，6个3D地图点空间位置的编号是P 1～P 6，6个3D地图点描述子的编号是F 1～F 6，对应表如表1所示。
表1
Figure PCTCN2021105624-appb-000001
需要说明的是，表1是区域描述子的编号和3D地图点的编号之间的对应表的一个示例，该对应表也可以采用其他的格式或方式呈现，本申请对此不做具体限定。
可选的，上述对应关系也可以隐式的用区域描述子和3D地图点的存储位置进行描述，例如，在内存中先存储T1，跟着存储P 1、P 2和P 3的数据，接着存储T2，跟着存储P 2和P 3的数据，最后存储T3，跟着存储P 3、P 4、P 5和P 6的数据。
图2为本申请实施例提供的电子设备20的一种结构示意图，如图2所示，电子设备20可以是图1所示实施例中的第一电子设备和一个或多个第二电子设备中的至少一者。应该理解的是，图2所示的结构并不构成对电子设备20的具体限定。在本申请另一些实施例中，电子设备20可以包括比图2所示结构更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图2中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
电子设备20可包括：芯片21、存储器22(一个或多个计算机可读存储介质)、 用户接口23、显示组件24、摄像头25、传感器26、用于设备定位的定位模块27以及用于通信的收发器28。这些部件之间可以通过一个或多个总线29通信。
芯片21可以集成包括：一个或多个处理器211、时钟模块212以及电源管理模块213。集成于芯片21中的时钟模块212主要用于为处理器211提供数据传输和时序控制所需要的计时器，计时器可以实现数据传输和时序控制的时钟功能。处理器211可以根据指令操作码和时序信号，执行运算，产生操作控制信号，完成取指令和执行指令的控制。集成于芯片21中的电源管理模块213主要用于为芯片21以及电子设备20的其他部件提供稳定的、高精确度的电压。
处理器211又可称为中央处理器(central processing unit，CPU)，处理器211具体可以包括一个或多个处理单元，例如，处理器211可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或，神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
在一种可能的实现方式中，处理器211可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或，通用串行总线(universal serial bus，USB)接口等。
存储器22可以和处理器211通过总线29连接，也可以与处理器311耦合在一起，用于存储各种软件程序和/或多组指令。存储器22可以包括高速随机存取的存储器(例如高速缓冲存储器)，也可以包括非易失性存储器，例如一个或多个磁盘存储设备、闪存设备或其他非易失性固态存储设备。存储器22可以存储操作系统，例如，安卓(Android)，苹果移动平台(IOS)，微软窗口操作系统(Windows)或者Linux等嵌入式操作系统。存储器22还可以存储数据，例如，图像数据、点云数据、3D地图的数据、位姿数据、坐标系转换信息、地图更新信息等。存储器22还可以存储计算机可执行的程序代码，该计算机可执行的程序代码包括指令，例如，通信程序指令，SLAM系统的相关程序指令等。存储器22还可以存储一个或多个应用程序，例如，AR/VR/MR等虚拟场景应用程序、地图类应用程序、图像管理类应用程序、导航与控制类应用程序等。存储器22还可以存储用户界面程序，该用户界面程序可以通过图形化的操作界面将应用程序的内容，例如，AR/VR/MR等虚拟场景中的虚拟物体，形象逼真的显示出来并通过显示组件24呈现，以及实现通过菜单、对话框和按键等输入控件接收用户对应用程序的控制操作。
用户接口23例如可以是触控面板，该触控面板可以检测用户在其上的操作指令，用户接口23也例如可以是小键盘、物理按键或者鼠标等。
电子设备20可以包括一个或多个显示组件24。电子设备20可以通过显示组件 24、芯片21中的图形处理器(GPU)和应用处理器(AP)等共同实现显示功能。GPU是实现图像处理的微处理器，其连接显示组件24和应用处理器，GPU执行数学和几何计算以用于图形渲染。显示组件24可以显示电子设备20输出的界面内容，例如，显示AR/VR/MR等虚拟场景中的图像、视频等，界面内容可包括正在运行的应用程序的界面以及系统级别菜单等，具体可以由下述界面元素组成：输入型界面元素，例如按键(Button)，文本输入框(Text)，滑动条(Scroll Bar)，菜单(Menu)等；输出型界面元素，例如视窗(Window)，标签(Label)，图像，视频，动画等。
显示组件24可以是显示面板，镜片(例如VR眼镜)，投影屏等。显示面板也可以称为显示屏，例如，可以是触摸屏、柔性屏、曲面屏等，也可以是其它光学组件。应当理解的，本申请实施例中的电子设备具有的显示屏可以是触摸屏、柔性屏、曲面屏或者其它形式的屏幕，亦即电子设备的显示屏具有显示图像的功能，而对于显示屏的具体材质和形状不作具体限定。
例如，当显示组件24包括显示面板时，显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。此外，在一种可能的实现方式中，可以将用户接口23中的触控面板和显示组件24中的显示面板耦合在一起，例如，触控面板可设置于显示面板下方，触控面板用于检测用户通过显示面板输入触控操作(例如点击、滑动，触摸等)时作用于显示面板上的触控压力，显示面板用于进行内容显示。
摄像头25可以是单目摄像头或双目摄像头或深度摄像头，用于对环境进行拍摄/录像，以获得图像/视频图像。摄像头25所采集的图像/视频图像例如可作为SLAM系统的一种输入数据，或者可通过显示组件24进行图像/视频显示。
在一种可能的实现方式中，也可以把摄像头25视为一种传感器。摄像头25采集的图像可以是IMG格式，也可以是其他格式类型，本申请实施例对此不做具体限定。
传感器26可以用于采集与电子设备20的状态变化(例如，旋转、摆动、移动、抖动等)相关的数据，传感器26所采集的数据可以作为SLAM系统的一种输入数据。传感器26可以包括一种或多种传感器，例如，惯性测量单元(inertial measurement unit，IMU)，飞行时间(time of flight，TOF)传感器等。IMU中可以包括陀螺仪、加速度计等传感器，陀螺仪用于测量电子设备运动时的角速度，加速度计用于测量电子设备运动时的加速度。TOF传感器中可以包括光发射器和光接收器，光发射器用于向外发射光线，例如，激光、红外线、雷达波等，光接收器用于检测反射的光线，例如，反射的激光、红外线、雷达波等。
需要说明的，传感器26还可以包括更多的其他传感器，例如惯性传感器，气压计，磁力计，轮速计等，本申请实施例对此不做具体限定。
定位模块27用于实现对电子设备20的物理定位，例如，用于获取电子设备20 的初始位置。定位模块27可以包括WiFi定位模块、蓝牙定位模块、基站定位模块、卫星定位模块中的一个或多个。卫星定位模块中可以设置有全球导航卫星系统(global navigation satellite system，GNSS)以辅助定位，GNSS不限于北斗系统、全球定位系统(global positioning system，GPS)系统、格洛纳斯(global navigation satellite system，GLONASS)系统、伽利略卫星导航系统(Galileo)系统。
收发器28用于实现电子设备20与其它设备(例如，服务器、其它电子设备等)之间的通信。收发器28集成了发射器和接收器，分别用于发送和接收射频信号。具体实现中，收发器28包括但不限于：天线系统、射频(radio frequency，RF)收发器、一个或多个放大器、调谐器、一个或多个振荡器、数字信号处理器、编译码器(CODEC)芯片、用户身份识别卡(subscriber identification module，SIM)卡和存储介质等。在一种可能的实现方式中，还可以在单独的芯片上实现收发器28。收发器28支持2G/3G/4G/5G等中的至少一种数据网络通信，和/或，支持以下近距无线通信的方式中的至少一种：蓝牙(bluetooth，BT)通信，无线保真(wirelessfidelity，WiFi)通信，近场通信(near field communication，NFC)，红外(infrared，IR)无线通信，超宽带(ultra wide band，UWB)通信，紫蜂协议(ZigBee)通信。
本申请实施例中，处理器211通过运行存储在存储器22的程序代码，从而执行电子设备20的各种功能应用和数据处理。
图3为本申请实施例提供的服务器30的一种结构示意图，如图3所示，服务器30可以是图1所示实施例中的服务器。服务器30包括处理器301、存储器302(一个或多个计算机可读存储介质)和收发器303。这些部件之间可以通过一个或多个总线304通信。
处理器301可以是一个或多个CPU，在处理器301是一个CPU的情况下，该CPU可以是单核CPU，也可以是多核CPU。
存储器302可以和处理器301通过总线304连接，也可以和处理器301耦合在一起，用于存储各种程序代码和/或多组指令、以及数据(例如，地图数据、位姿数据等)。具体实现中，存储器302包括但不限于是随机存储记忆体(Random Access Memory，RAM)、只读存储器(Read-Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、或便携式只读存储器(Compact Disc Read-Only Memory，CD-ROM)等。
收发器303主要集成了接收器和发射器，其中接收器用于接收电子设备发送的数据(例如，请求、图像等)，发射器用于向电子设备发送数据(例如，地图数据、位姿数据等)。
应当理解，图3所示的服务器30仅为本申请实施例提供的一个示例，服务器30还可以具有比图示更多的部件，本申请实施例对此不做具体限定。
本申请实施例中，处理器301通过运行存储在存储器302中的程序代码，从而执行服务器30的各种功能应用和数据处理。
本申请实施例中使用的术语“耦合”意指直接连接或者通过一个或多个居间组件或电路连接。
图4a为本申请实施例提供的一个应用场景的示意图，如图4a所示，该应用场景 是电子设备通过传感器采集视觉信息，结合视觉信息以及来自服务器的3D地图确定电子设备的当前位姿。
3D地图是由服务器提供的，即服务器创建3D地图，然后对该3D地图进行压缩处理，将3D地图的压缩数据传输给电子设备；电子设备收到3D地图的压缩数据后进行解压缩处理，得到3D地图的重构数据，结合采集的视觉信息以及3D地图的重构数据确定电子设备的当前位姿。位姿即电子设备的位置信息，可以是在世界坐标系中的绝对位姿，也可以是相对于环境中某个点的相对位姿。
本申请实施例中，服务器可以预先创建3D地图，将3D地图进行压缩处理后存储在本地，这样可以节省存储空间。另外，服务器也可以将3D地图的压缩数据传输给其他设备，例如，云存储器。
1、服务器创建3D地图，并压缩得到3D地图的压缩数据存储在本地。
服务器对3D地图进行压缩处理可以节省本地的存储空间。
2、电子设备向服务器发送地图下载请求，该地图下载请求的触发方式有两种：
(1)用户打开安装于电子设备上的地图类应用程序，该应用程序向其对应的服务器上传基于GPS定位或者WiFi定位得到的位置信息，该上传操作可以触发地图下载请求。由于上传的内容包括位置信息，因此服务器可以基于位置信息进行一个初步估计，将位置信息指示的定位点所属区域的3D地图的压缩数据传输给电子设备。位置信息指示的定位点所属区域的范围可以预先设定，例如，该所属区域可以是定位点所在的各个级别的行政区(包括县、市、国家或者行政区等)，也可以是以定位点为中心、设定距离为半径的圆周区域。
(2)用户打开安装于电子设备上的地图类应用程序，在该应用程序中主动输入或者选择一个区域，例如，用户主动输入“xx商业中心”，或者从一个列表“A街道、B街道、C街道”中选择了“A街道”。用户的前述操作可以触发地图下载请求。无论是用户输入或是选择包含了一个地理位置，因此服务器将该地理位置的3D地图的压缩数据传输给电子设备。
应当理解的是，本申请实施例中除了上述两种方式还可以采用其他方式触发地图下载请求，例如，电子设备自动检测是否符合下载3D地图或者启动下载3D地图的条件，或者，电子设备通过检测环境光线的变化或者检测环境的变化来启动下载3D地图，以向服务器请求下载一个区域范围内的3D地图，区域范围的大小不做具体限定。
3、服务器向电子设备发送3D地图的压缩数据。
4、电子设备通过传感器采集视觉信息。
需要说明的是，步骤3和步骤4是相互独立的，不限定先后顺序。
5、电子设备对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
6、电子设备根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
电子设备在收到3D地图的压缩数据后，不需要立即解压缩，只需要在基于视觉信息进行定位之前解压缩得到3D地图的重构数据即可。例如，用户可以以下载“离线地图”的方式预先下载一个区域范围的3D地图的压缩数据，而在需要定位时，才 对3D地图的压缩数据进行解压缩。
图4b为本申请实施例提供的一个应用场景的示意图，如图4b所示，该应用场景是电子设备通过传感器采集视觉信息，服务器结合来自电子设备的视觉信息以及3D地图确定电子设备的当前位姿。
3D地图是由服务器提供的，即服务器创建3D地图，然后对该3D地图进行压缩处理，将3D地图的压缩数据存储在本地。当收到来自电子设备的视觉信息时，服务器进行解压缩处理，得到3D地图的重构数据，结合视觉信息以及3D地图的重构数据确定电子设备的当前位姿。
1、服务器创建3D地图，并压缩得到3D地图的压缩数据存储在本地。
2、电子设备通过传感器采集视觉信息。
3、电子设备向服务器发送视觉信息。
4、服务器对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
应当理解的，服务器对3D地图进行压缩处理是为了节省存储空间。
5、服务器根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
6、服务器向电子设备发送位姿。
图4c为本申请实施例提供的一个应用场景的示意图，如图4c所示，该应用场景是电子设备通过传感器采集视觉信息，结合采集的视觉信息以及3D地图确定电子设备的当前位姿。
3D地图是由电子设备提供的，即电子设备创建3D地图，然后对该3D地图进行压缩处理，将3D地图的压缩数据存储在本地。当采集到的视觉信息时，电子设备进行解压缩处理，得到3D地图的重构数据，结合采集的视觉信息以及3D地图的重构数据确定电子设备的当前位姿。
1、电子设备创建3D地图，并压缩得到3D地图的压缩数据存储在本地。
应当理解的，电子设备对3D地图进行压缩处理是为了节省存储空间。
2、电子设备通过传感器采集得到视觉信息。
3、电子设备对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
4、电子设备根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
图4d为本申请实施例提供的一个应用场景的示意图，如图4b所示，该应用场景是第二电子设备通过传感器采集视觉信息，结合视觉信息以及来自服务器的3D地图确定第二电子设备的当前位姿。
3D地图是由第一电子设备创建的，即第一电子设备创建3D地图，对该3D地图进行压缩处理，然后将3D地图的压缩数据发送给服务器，服务器再把3D地图的压缩数据发送给第二电子设备，第二电子设备进行解压缩处理，得到3D地图的重构数据，结合采集的视觉信息以及3D地图的重构数据确定第二电子设备的当前位姿。
本申请实施例中，第一电子设备可以预先创建3D地图，将3D地图进行压缩处理后传输给服务器，这样可以减少传输带宽。
1、第一电子设备创建3D地图，并压缩得到3D地图的压缩数据。
2、第一电子设备向服务器发送3D地图的压缩数据。
第一电子设备对3D地图进行压缩处理后再传输，可以减少传输带宽，提高传输效率。
3、第二电子设备向服务器发送地图下载请求。
第二电子设备发送地图下载请求也可以基于图4a所示的触发方式。
4、服务器向第二电子设备发送3D地图的压缩数据。
5、第二电子设备对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
6、第二电子设备通过传感器采集得到视觉信息。
7、第二电子设备根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
图4e为本申请实施例提供的一个应用场景的示意图，如图4c所示，该应用场景是第二电子设备通过传感器采集视觉信息，服务器结合来自第二电子设备的视觉信息以及来自第一电子设备的3D地图确定第二电子设备的当前位姿。
3D地图是由第一电子设备创建的，即第一电子设备创建3D地图，对该3D地图进行压缩处理，然后将3D地图的压缩数据发送给服务器。服务器进行解压缩处理，得到3D地图的重构数据，结合来自第二电子设备的视觉信息以及3D地图的重构数据确定第二电子设备的当前位姿。
1、第一电子设备创建3D地图，并压缩得到3D地图的压缩数据。
2、第一电子设备向服务器发送3D地图的压缩数据。
3、第二电子设备通过传感器采集得到视觉信息。
4、第二电子设备向服务器发送定位请求，该定位请求中携带视觉信息。
5、服务器对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
6、服务器根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
7、服务器向第二电子设备发送定位得到的位姿。
图4f为本申请实施例提供的一个应用场景的示意图，如图4d所示，该应用场景是第二电子设备通过传感器采集视觉信息，结合视觉信息以及来自第一电子设备的3D地图确定第二电子设备的当前位姿。
3D地图是由第一电子设备创建的，即第一电子设备创建3D地图，对该3D地图进行压缩处理，然后将3D地图的压缩数据发送给第二电子设备，第二电子设备进行解压缩处理，得到3D地图的重构数据，结合采集的视觉信息以及来自第一电子设备的3D地图确定第二电子设备的当前位姿。
1、第一电子设备创建3D地图，并压缩得到3D地图的压缩数据存储在本地。
2、第二电子设备向第一电子设备发送地图下载请求。
3、第一电子设备向第二电子设备发送3D地图的压缩数据。
4、第二电子设备对3D地图的压缩数据进行解压缩以得到3D地图的重构数据。
5、第二电子设备通过传感器采集得到视觉信息。
6、第二电子设备根据视觉信息在3D地图中进行定位以得到与视觉信息对应的位姿。
图4a～图4f所示实施例中，所采用的定位算法可以包括：
(1)从视觉信息中提取待检索区域描述子，提取待检索区域描述子使用的算法 和从3D地图中提取区域描述子的算法一致。
(2)从视觉信息中提取待检索3D地图点，并获取待检索3D地图点空间位置和待检索3D地图点描述子，提取待检索3D地图点描述子的算法和从3D地图中提取3D地图点描述子的算法一致。
(3)根据待检索区域描述子在3D地图的数据包含的多个区域描述子中进行检索以得到多个候选区域描述子。
本申请实施例中，可以计算待检索区域描述子分别与多个区域描述子中的每一个区域描述子的距离，该距离可以包括汉明距离、曼哈顿距离或者欧氏距离等，然后选择符合条件(例如距离小于阈值)的至少一个区域描述子，即为候选区域描述子。
(4)使用待检索3D地图点描述子分别与多个候选区域描述子所对应的3D地图点描述子进行匹配，匹配亦即计算待检索3D地图点描述子分别与多个候选区域描述子对应的3D地图点描述子的相似度，找出最相似的3D地图点。
(5)根据匹配到的3D地图点，使用透视n点相机姿态估计(pespective-n-point，PnP)、高效透视n点相机姿态估计(efficient perspective-n-point camera pose estimation，EPnP)等位姿求解算法计算得到电子设备的位姿。
在图4a～图4f任一应用场景中，基于本申请实施例的3D地图进行定位，以得到电子设备当前的位姿，该位姿可以应用于AR导航、AR人机交互、辅助驾驶、自动驾驶等领域。以基于该位姿进行AR导航为例，图4g为本申请实施例提供的电子设备显示的一种用户界面的示意图，电子设备可以基于该位姿，显示如图4g所示的用户界面，该用户界面上可以包括导航前往会议室2的导航箭头指示，该导航前往会议室2的导航箭头指示可以是基于该位姿从服务器获取的或者从本地获取的虚拟物体。该用户界面上还可以包括传感器采集的视觉信息，例如，如图4g所示的建筑物。用户参照电子设备的如图4g所示的用户界面，前往会议室2。
需要说明的是，本申请实施例所涉及的解压缩得到的3D地图的重构数据，也可以称之为3D地图的重建数据。
而图4a～图4f所示实施例中，均涉及到对3D地图进行压缩处理和解压缩处理，以及基于解压缩得到的3D地图的重构数据进行定位，以得到电子设备当前的位姿。对于定位，一种可实现方式，可以基于完全解压缩得到的3D地图的重构数据进行定位。另一种可实现方式，可以使用本申请下述实施例提供的3D地图的检索方法，筛选出描述子，利用描述子进行定位。本申请下述实施例提供3D地图的检索方法，通过将检索和解压缩或从压缩数据提取交替进行，以在3D地图中进行多级检索，得到描述子。相较于在完全解压得到的3D地图的重构数据中进行检索，本申请下述实施例提供的3D地图的检索方法的至少两级检索中存在一级非完全解压缩检索，可以提升检索速度，至少两级检索中另一级检索可以保证检索准确率。非完全解压缩是指未完全解压缩得到3D地图的全部重构数据。以下对该3D地图的检索方法进行说明。
本申请实施例所涉及的3D地图的检索是指，在3D地图的多个3D地图点中找到最相似或最相关的一些3D地图点。
本申请实施例所涉及的二值化数据的长度可以是二值化数据的比特位个数(也称比特数)。例如，二值化数据为110，则该二值化数据的比特位个数为3。
需要说明的是，本申请实施例提供的检索方法也可以应用于其他技术领域，例如，图像或视频数据的检索，音频数据的检索、点云数据的检索等。
基于上文的描述，下面给出本申请实施例提供的3D地图的检索方法。对于下文描述的各方法实施例，为了方便起见，将其都表述为一系列的动作步骤的组合，但是本领域技术人员应该知悉，本申请技术方案的具体实现并不受所描述的一系列的动作步骤的顺序的限制。
参见图5A，图5A是本申请实施例提供的一种3D地图的检索方法的片段流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也可以应用于图1至图4f任一所示例的服务器。该方法包括但不限于以下步骤：
S101A、从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，S个3D地图描述子对应3D地图中的多个3D地图点。
该二值化数据可以是第一二值化数据、第二二值化数据、……第n二值化数据等，n为正整数。应当理解的是，第一二值化数据、第二二值化数据和第三二值化数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第n二值化数据为例进行解释说明。
从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第n二值化数据。
S个3D地图描述子的第n二值化数据是从S个3D地图描述子的压缩数据中提取的。S为正整数。S表示第i级检索中需要检索的3D地图描述子的个数。S＜T，T表示3D地图中的3D地图描述子的总个数。
其中，S个3D地图描述子对应的多个3D地图点的个数可以大于S，也可以等于S。
例如，S个3D地图描述子可以是S个区域描述子，或者S个3D地图点描述子。其中，一个区域描述子对应多个3D地图点，一个3D地图点描述子对应一个3D地图点，其具体解释说明可以参见前述实施例的解释说明，此处不再赘述。
S个3D地图描述子的压缩数据可以是S个区域描述子或S个3D地图点描述子经过压缩处理得到的。这里的压缩处理可以包括：紧凑化处理。其中，紧凑化处理可以包括二值化处理和量化处理。S个3D地图描述子的第n二值化数据可以是S个3D地图描述子各自的压缩数据的部分数据，例如，S个3D地图描述子的第n二值化数据可以是S个3D地图描述子各自的压缩数据的部分或全部二值化数据。
该二值化处理包括但不限于：迭代量化(iterative quantization，ITQ)的哈希、局部敏感哈希(locality-sensitive hashing，LSH)、或谱哈希方法(spectral hashing)等。示例性的，通过二值化处理可以将描述子映射到汉明空间(也称为二值空间)中，得到二值化数据。
该量化处理包括但不限于：标量量化、矢量量化、或乘积量化等。示例性的，通过量化处理可以将3D地图描述子压缩为一个或多个量化索引，该一个或多个量化索引即为该3D地图描述子的量化数据。其中，一个或多个量化索引中的各个量化索引对应于一个量化中心。
可选的，本申请实施例的执行主体可以对S个3D地图描述子的压缩数据进行熵解码，从中提取S个3D地图描述子的第n二值化数据。
S102A、根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子。
以第n二值化数据为例进行解释说明。
根据检索描述子的第n二值化数据，在S个3D地图描述子的第n二值化数据中进行第i级检索，以得到P个3D地图描述子。
该检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。
这里的检索描述子可以是上述实施例中的待检索区域描述子，或者，待检索3D地图点描述子。也即，该检索描述子可以是从视觉信息中提取的待检索区域描述子，该待检索区域描述子用于描述多个3D地图点中的部分或全部3D地图点的特征。该待检索区域描述子可以是一条向量，例如，128维的向量，或512维的向量等等。待检索区域描述子的维数不以上述举例作为限制，其还可以是其他数值，本申请实施例不一一举例说明。或者，该检索描述子可以是从视觉信息中提取的待检索3D地图点描述子，该待检索3D地图点描述子用于表示对应3D地图点的局部特征。该待检索3D地图点描述子可以是一条向量，该向量的维数可以是任意正整数。
获取检索描述子的方式可以包括但不限于如下几种方式。一种可实现方式，本实施例的执行主体可以接收其他设备发送的检索描述子。例如，本实施例的执行主体可以为如图1所示的第一电子设备，第一电子设备可以接收第二电子设备发送的检索描述子。又例如，本实施例的执行主体可以为如图1所示的服务器，服务器可以接收第一电子设备或第二电子设备发送的检索描述子。另一种可实现方式，本实施例的执行主体可以接收其他设备发送的视觉信息，从视觉信息中提取检索描述子。例如，本实施例的执行主体可以为如图1所示的第一电子设备，第一电子设备可以接收第二电子设备发送的视觉信息，第一电子设备从视觉信息中提取检索描述子。又例如，本实施例的执行主体可以为如图1所示的服务器，服务器可以接收第一电子设备或第二电子设备发送的视觉信息，服务器从视觉信息中提取检索描述子。又一种可实现方式，本实施例的执行主体可以检测用户输入的视觉信息采集的操作，响应于该操作，触发自身传感器对真实环境进行视觉信息采集，得到视觉信息，从视觉信息中提取检索描述子。例如，本实施例的执行主体可以为如图1所示的第一电子设备，当第一电子设备检测到用户输入的视觉信息采集的操作时，响应于该操作，触发自身传感器对真实环境进行视觉信息采集，得到视觉信息，从视觉信息中提取检索描述子。该视觉信息采集的操作可以是通过点击、触摸、滑动、或抖动等方式启动传感器采集视觉信息，也可以是声控或者其它途径启动传感器采集视觉信息，本申请实施例对此不作限定。举例来说，第一电子设备检测到用户对用户界面上的拍摄按钮的触摸操作后，响应于该触摸操作，第一电子设备的摄像头被启动等等。
检索描述子的第n二值化数据可以是检索描述子的部分或全部二值化数据。一种示例，本实施例的执行主体可以对检索描述子进行二值化处理，得到检索描述子的二值化数据，从检索描述子的二值化数据中选取(也称为提取)部分或全部作为检索描述子的第n二值化数据。
本实施例的执行主体可以使用检索描述子的第n二值化数据，在S个3D地图描述子 的第n二值化数据中进行检索，以得到与检索描述子的第n二值化数据最相似或最相关的P个3D地图描述子。
P为正整数，P表示第i级检索筛选出的3D地图描述子的个数。P＜T。
在一些实施例中，P<S。示例性的，通过第i级检索得到一个3D地图描述子集合，该3D地图描述子集合中的3D地图描述子的个数少于第i级检索前的3D地图描述子集合中的3D地图描述子的个数。在一些实施例中，P可以大于S。其具体解释说明可以参见下述图7所示实施例的解释说明，此处不再赘述。
当P<S，即P个3D地图描述子属于S个3D地图描述子，P个3D地图描述子各自的第n二值化数据与检索描述子的第n二值化数据的相似度或相关度，高于S个3D地图描述子中除了该P个3D地图描述子之外的其他3D地图描述子。其中，相似度或相关度的计算方式可以有很多实现方式，例如，通过计算两个3D地图描述子(3D地图描述子和检索描述子)各自的第n二值化数据的第一距离，确定两个3D地图描述子各自的第n二值化数据的相关度。其中，第一距离越小相关度越高。
本申请实施例中第一距离可以包括汉明距离等利用二值化数据得到的距离。例如，第一距离可以是汉明距离。又例如，第一距离可以是对两个二值化数据的对应位取差值的绝对值，并将所有差值的绝对值相加得到的一种距离。基于第一距离的检索指的是通过计算第一距离，确定相关度或相似度，以筛选3D地图描述子。
S103A、对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的重构数据。
该重构数据可以是第一重构数据、第二重构数据、……第m重构数据等，m为正整数。应当理解的是，第一重构数据、第二重构数据和第三重构数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第m重构数据为例进行解释说明。
对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的第m重构数据。
通过S101A和S102A可以筛选得到P个3D地图描述子，通过S103A可以对这P个3D地图描述子的压缩数据进行第m级解压缩，得到这P个3D地图描述子的第m重构数据，以进一步基于第m重构数据进行检索。m为正整数。第m级解压缩可以包括反量化处理。第m级解压缩可以是阶段性解压缩(即完全解压缩的一部分)，或者，也可以是完全解压缩。
P个3D地图描述子的第m重构数据可以是P个3D地图描述子各自的压缩数据的部分或全部数据经过第m级解压缩得到的。例如，P个3D地图描述子的第m重构数据可以是对P个3D地图描述子各自的压缩数据的部分或全部经过一种反量化处理得到的反量化数据。需要说明的是，该反量化数据也可以称之为重构数据，该重构数据可以是通过已有技术中的反量化方式得到。又例如，P个3D地图描述子的第m重构数据可以是对P个3D地图描述子各自的量化数据(压缩数据的一部分)，进行反量化处理，得到P个反量化数据，之后再根据P个反量化数据和P个3D地图描述子各自的二值化数据(压缩数据的一部分)得到的。
在一些实施例中，第m级解压缩可以还可以包括熵解码、预测等其他解压缩处理，本申请实施例不一一举例说明。
S104A、根据检索描述子的部分数据或全部数据，在P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子。
Q为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
该重构数据可以是第一重构数据、第二重构数据、……第m重构数据等，m为正整数。应当理解的是，第一重构数据、第二重构数据和第三重构数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第m重构数据为例进行解释说明。
检索描述子的部分数据或全部数据是指检索描述子的部分或全部分量。检索描述子的分量是指检索描述子这个向量中的一个维度的数值。例如，该检索描述子可以是一条向量，例如，M维的向量，M维中的每一维是该向量的一个分量。
与S102A类似，本实施例的执行主体可以使用检索描述子的部分数据或全部数据，在P个3D地图描述子的第m重构数据中进行检索，以得到与检索描述子的第m重构数据最相似或最相关的Q个3D地图描述子。Q可以小于P。也可以理解为，通过第j级检索得到另一个3D地图描述子集合，该3D地图描述子集合中的3D地图描述子的个数少于第i级检索得到的3D地图描述子的个数。
Q个3D地图描述子各自的第m重构数据与检索描述子的部分数据或全部数据的相似度或相关度，高于P个3D地图描述子中除了该Q个3D地图描述子之外的其他3D地图描述子。其中，相似度或相关度的计算方式可以有很多实现方式，例如，通过计算两个3D地图描述子(3D地图描述子的第m重构数据和检索描述子的部分或全部数据)的第二距离，确定两个3D地图描述子的相关度。第二距离可以包括但不限于欧式距离、内积距离、余弦距离、曼哈顿距离等。例如，欧式距离越小相关度越高，内积距离越大相关度越高。
当然可以理解的，在S104A之后可以执行一个条件判断步骤，在满足条件时，可以结束本实施例的检索，在不满足条件时，可以继续执行一级或多级检索。这个条件可以是任意条件，其可以根据需求进行合理设置，例如，条件可以是相似度或相关度高于或等于预设阈值，也可以是检索得到的3D地图描述子的个数小于或等于预设个数等。又例如，条件可以是检索得到用于定位的下述N个3D地图描述子。
本申请实施例的3D地图的检索方法是一个片段流程，在该片段流程之前和/或之后还可以包括其他级检索，换言之，3D地图的检索方法可以包括比图5A所示的更多级数的多级检索，例如，3级、4级或5级等等。例如，如图5A所示实施例的S101A之前还可以有更多级的检索，或者，如图5A所示实施例的S104A之后还可以有更多级的检索，此处不一一举例说明。
在一些实施例中，通过多级检索得到的Q个3D地图描述子可以用于定位。若Q个3D地图描述子中的N个3D地图描述子对应的3D地图点与检索描述子对应的3D地图点匹配，则可以根据N个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿信息。该电子设备可以是采集上述视觉信息的电子设备。0＜N≤Q。
本实施例，从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据， 根据检索描述子的二值化数据，在S个3D地图描述子的二值化数据中进行第i级检索，筛选得到P个3D地图描述子。对P个3D地图描述子的压缩数据进行第m级解压缩，得到P个3D地图描述子的重构数据，根据检索描述子的部分数据或全部数据，在P个3D地图描述子的重构数据中进行第j级检索，以筛选出更少个数的3D地图描述子。至少两级检索中的一级检索使用压缩数据中的二值化数据，可以提升检索速度，另一级检索使用解压缩得到的重构数据，可以保证检索准确性。相较于在完全解压得到的3D地图的重构数据中进行检索，本申请实施例提供的3D地图的检索方法可以提升检索性能。
参见图5B，图5B是本申请实施例提供的一种3D地图的检索方法的片段流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也可以应用于图1至图4f任一所示例的服务器。该方法包括但不限于以下步骤：
S101B、从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第一二值化数据，S个3D地图描述子对应3D地图中的多个3D地图点。
该第一二值化数据可以指代第一二值化数据、第二二值化数据、……第n二值化数据等任意一个，n为正整数。应当理解的是，第一二值化数据、第二二值化数据和第三二值化数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第n二值化数据为例进行解释说明。
从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第n二值化数据，S个3D地图描述子对应3D地图中的多个3D地图点。
S102B、根据检索描述子的第一二值化数据，在S个3D地图描述子的第一二值化数据中进行第i级检索，以得到P个3D地图描述子。
以下以第n二值化数据为例进行解释说明。可以根据检索描述子的第n二值化数据，在S个3D地图描述子的第n二值化数据中进行第i级检索，以得到P个3D地图描述子。
其中，S101B和S102B的具体解释说明可以参见图5A所示实施例的S101A和S102A。
S103B、从P个3D地图描述子的压缩数据中提取P个3D地图描述子的第二二值化数据。
该第二二值化数据可以指代第二二值化数据、第三二值化数据、……第q二值化数据等任意一个，q为正整数。应当理解的是，第二二值化数据和第三二值化数据等中的“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第q二值化数据为例进行解释说明。一种示例，q＝n+1。
通过S101B和S102B可以从S个3D地图描述子中筛选得到P个3D地图描述子，再从这P个3D地图描述子的压缩数据中提取P个3D地图描述子的第q二值化数据，以进一步基于第q二值化数据进行检索。P个3D地图描述子的第q二值化数据可以是P个3D地图描述子各自的压缩数据的部分或全部数据，例如，P个3D地图描述子的第q二值化数据可以是P个3D地图描述子各自的部分或全部二值化数据。
S个3D地图描述子的各个3D地图描述子的第n二值化数据，与P个3D地图描述子的各个3D地图描述子的第q二值化数据不同。一种可实现方式，该不同可以是长度不同，即S个3D地图描述子的各个3D地图描述子的第n二值化数据的长度，与P个3D地图描述子的各个3D地图描述子的长度不同。另一种可实现方式，该不同可以是第n二值化数据和第 q二值化数据所对应的二值化处理方式不同，即S个3D地图描述子的各个3D地图描述子的第n二值化数据所对应的二值化处理方式，与P个3D地图描述子的各个3D地图描述子所对应的二值化处理方式不同。例如，S个3D地图描述子的各个3D地图描述子的第n二值化数据是经一种二值化处理方式(例如，局部敏感哈希)得到的，P个3D地图描述子的各个3D地图描述子是经另一种二值化处理方式(例如，谱哈希)得到的。
第n二值化数据的长度，可以用第n二值化数据的比特数来衡量或表示。类似的，第q二值化数据的长度，可以用第q二值化数据的比特数来衡量或表示。
当P个3D地图描述子属于S个3D地图描述子(P<S)时，P个3D地图描述子的各个3D地图描述子的第n二值化数据在3D地图描述子的压缩数据中的位置，不同于该3D地图描述子的第q二值化数据在该3D地图描述子的压缩数据中的位置。该位置可以是起始位置或结束位置等。
一种示例，P个3D地图描述子的各个3D地图描述子的第n二值化数据所对应的原始3D地图描述子的分量，与相应的3D地图描述子的第q二值化数据所对应的原始3D地图描述子的分量不同。即一个3D地图描述子的第n二值化数据可以是该3D地图描述子的第一部分(例如，3D地图描述子的前128维的分量)的二值化数据，该3D地图描述子的第q二值化数据可以是该3D地图描述子的第二部分(例如，3D地图描述子的后128维的分量)的二值化数据。
当P个3D地图描述子属于S个3D地图描述子(P<S)时，P个3D地图描述子的各个3D地图描述子的第n二值化数据的长度，可以小于P个3D地图描述子的各个3D地图描述子的第q二值化数据的长度。第i级检索中的各个3D地图描述子的第n二值化数据的长度较短，可以提升检索速度，第j级检索中的3D地图描述子的第q二值化数据的长度较长，可以提升检索准确率。通过差异化的至少两级检索，以实现检索速度和检索准确率的综合提升。或者，P个3D地图描述子的各个3D地图描述子的第n二值化数据的长度，可以大于P个3D地图描述子的各个3D地图描述子的第q二值化数据的长度。或者，P个3D地图描述子的各个3D地图描述子的第n二值化数据的长度，可以等于P个3D地图描述子的各个3D地图描述子的第q二值化数据的长度。
S104B、根据检索描述子的第二二值化数据，在P个3D地图描述子的第二二值化数据中进行第j级检索，以得到Q个3D地图描述子。
以下以第q二值化数据为例进行解释说明。
根据检索描述子的第q二值化数据，在P个3D地图描述子的第q二值化数据中进行第j级检索，以得到Q个3D地图描述子。
S、P、Q、i和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
检索描述子的第q二值化数据可以是检索描述子的部分或全部二值化数据。一种示例，本实施例的执行主体可以对检索描述子进行二值化处理，得到检索描述子的二值化数据，从检索描述子的二值化数据中选取(也称为提取)部分或全部作为检索描述子的第n+1表示。需要说明的是，检索描述子的第n二值化数据与检索描述子的第q二值化数据可以相同，也可以不同。
与S102B类似，本实施例的执行主体可以使用检索描述子的第q二值化数据，在P个3D地图描述子的第q二值化数据中进行检索，以得到与检索描述子的第q二值化数据最相似或最相关的Q个3D地图描述子。Q可以小于P。也可以理解为，通过第j级检索得到另一个3D地图描述子集合，该3D地图描述子集合中的3D地图描述子的个数少于第i级检索得到的3D地图描述子的个数。
Q个3D地图描述子各自的第q二值化数据与检索描述子的第q二值化数据的相似度或相关度，高于P个3D地图描述子中除了该Q个3D地图描述子之外的其他3D地图描述子。其中，相似度或相关度的计算方式的解释说明可以参见S102B的相关解释说明，此处不再赘述。
可选的，上述S102B中的第i级检索和上述S104B中的第j级检索所采用的检索方式均为基于第一距离的检索，其具体解释说明可以参见图5A所示实施例的S102A的解释说明，此处不再赘述。
当然可以理解的，在S104B之后可以执行一个条件判断步骤，在满足条件时，可以结束本实施例的检索，在不满足条件时，可以继续执行一级或多级检索。这个条件可以是任意条件，其可以根据需求进行合理设置，例如，条件可以是相似度或相关度高于或等于预设阈值，也可以是检索得到的3D地图描述子的个数小于或等于预设个数等，本申请实施例不一一举例说明。
本申请实施例的3D地图的检索方法是一个片段流程，在该片段流程之前和/或之后还可以包括其他级检索，换言之，3D地图的检索方法可以包括比图5B所示的更多级数的多级检索，例如，3级、4级或5级等等。例如，如图5B所示实施例的S101B之前还可以有更多级的检索，或者，如图5B所示实施例的S104B之后还可以有更多级的检索，此处不一一举例说明。
在一些实施例中，通过多级检索得到的Q个3D地图描述子可以用于定位。若Q个3D地图描述子中的N个3D地图描述子对应的3D地图点与检索描述子对应的3D地图点匹配，则可以根据N个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿信息。该电子设备可以是采集上述视觉信息的电子设备。0＜N≤Q。
本实施例，从S个3D地图描述子的压缩数据中提取S个3D地图描述子的第一二值化数据，根据检索描述子的第一二值化数据，在S个3D地图描述子的第一二值化数据中进行第i级检索，筛选得到P个3D地图描述子。从P个3D地图描述子的压缩数据中提取P个3D地图描述子的第二二值化数据，根据检索描述子的第二二值化数据，在P个3D地图描述子的第二二值化数据中进行第j级检索，以筛选出更少个数的3D地图描述子。通过从压缩数据中提取一级检索的3D地图描述子的二值化数据，与一级检索交替进行，以在压缩状态的3D地图中进行至少两级检索，得到检索后的3D地图描述子。任意两级检索各自所使用的3D地图描述子的二值化数据所携带的信息内容不同或各自所使用的3D地图描述子的二值化数据的信息完整程度不同。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
参见图5C，图5C是本申请实施例提供的一种3D地图的检索方法的片段流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也 可以应用于图1至图4f任一所示例的服务器。该方法包括但不限于以下步骤：
S101C、对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的第一重构数据，S个3D地图描述子对应3D地图中的多个3D地图点。
该第一重构数据可以指代第一重构数据、第二重构数据、……第m重构数据等任意一个，m为正整数。应当理解的是，第一重构数据、第二重构数据和第三重构数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第一重构数据是第m重构数据为例进行解释说明。
对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的第m重构数据，S个3D地图描述子对应3D地图中的多个3D地图点。
S和m为正整数。S表示第i级检索中需要检索的3D地图描述子的个数。S＜T，T表示3D地图中的3D地图描述子的总个数。
其中，S个3D地图描述子对应的多个3D地图点的个数可以大于S，也可以等于S。
S个3D地图描述子的压缩数据可以是S个区域描述子或S个3D地图点描述子经过压缩处理得到的。这里的压缩处理可以包括：量化处理。可选的，该压缩处理还可以包括以下至少一项：二值化处理、预测处理或熵编码处理等。
S个3D地图描述子的第m重构数据是对S个3D地图描述子各自的压缩数据的部分数据进行第m级解压缩得到的。
m为正整数。第m级解压缩可以包括反量化处理。第m级解压缩可以是阶段性解压缩(即完全解压缩的一部分)。
S个3D地图描述子的第m重构数据可以是S个3D地图描述子各自的压缩数据的部分数据经过第m级解压缩得到的。例如，S个3D地图描述子的第m重构数据可以是对S个3D地图描述子各自的压缩数据的部分经过一种反量化处理得到的反量化数据。需要说明的是，该反量化数据也可以称之为重构数据，该重构数据可以是通过已有技术中的反量化方式得到。又例如，S个3D地图描述子的第m重构数据可以是对S个3D地图描述子各自的一部分量化数据(压缩数据的一部分)，进行反量化处理，得到S个反量化数据，之后再根据S个反量化数据和S个3D地图描述子各自的一部分二值化数据(压缩数据的一部分)得到的。
在一些实施例中，第m级解压缩可以还可以包括熵解码、预测等其他解压缩处理，本申请实施例不一一举例说明。
S102C、根据检索描述子的部分数据或全部数据，在S个3D地图描述子的第一重构数据中进行第i级检索，以得到P个3D地图描述子。
以下以第一重构数据是第m重构数据为例进行解释说明。
根据检索描述子的部分数据或全部数据，在S个3D地图描述子的第m重构数据中进行第i级检索，以得到P个3D地图描述子。
检索描述子的部分数据或全部数据是指检索描述子的部分或全部分量。检索描述子的分量是指检索描述子这个向量中的一个维度的数值。例如，该检索描述子可以是一条向量，例如，M维的向量，M维中的每一维是该向量的一个分量。
本实施例的执行主体可以使用检索描述子的部分数据或全部数据，在S个3D地图描述子的第m重构数据中进行检索，以得到与检索描述子的第m重构数据最相似或最相关的P个3D地图描述子。
在一些实施例中，P<S。示例性的，通过第i级检索得到一个3D地图描述子集合，该3D地图描述子集合中的3D地图描述子的个数少于第i级检索前的3D地图描述子集合中的3D地图描述子的个数。在一些实施例中，P可以大于S。其具体解释说明可以参见下述图7所示实施例的解释说明，此处不再赘述。
当P<S，即P个3D地图描述子属于S个3D地图描述子，P个3D地图描述子各自的第m重构数据与检索描述子的部分数据或全部数据的相似度或相关度，高于S个3D地图描述子中除了该P个3D地图描述子之外的其他3D地图描述子。其中，相似度或相关度的计算方式可以有很多实现方式，例如，通过计算两个3D地图描述子(3D地图描述子的第k重构数据和检索描述子的部分或全部数据)的第二距离，确定两个3D地图描述子的相关度。第二距离可以包括但不限于欧式距离、内积距离、余弦距离、曼哈顿距离等。例如，欧式距离越小相关度越高，内积距离越大相关度越高。
S103C、对P个3D地图描述子的压缩数据进行第k级解压缩，得到P个3D地图描述子的第二重构数据，S个3D地图描述子的压缩数据的解压缩过程包括第m级解压缩和第k级解压缩，或者，S个3D地图描述子的压缩数据的解压缩过程包括第k级解压缩，第k级解压缩包括第m级解压缩。
该第二重构数据可以指代第二重构数据、第三重构数据、……第k重构数据等任意一个，k为正整数，m<k。应当理解的是，第二重构数据和第三重构数据等中的“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以第二重构数据是第k重构数据为例进行解释说明。
对P个3D地图描述子的压缩数据进行第k级解压缩，得到P个3D地图描述子的第k重构数据，S个3D地图描述子的压缩数据的解压缩过程包括第m级解压缩和第k级解压缩，或者，S个3D地图描述子的压缩数据的解压缩过程包括第k级解压缩，第k级解压缩包括第m级解压缩。
通过S101C和S102C可以从S个3D地图描述子中筛选得到P个3D地图描述子，再对这P个3D地图描述子的压缩数据进行第k级解压缩得到P个3D地图描述子的第k重构数据，以进一步基于第k重构数据进行检索。在一些实施例中，k＝m+1。
P个3D地图描述子的第k重构数据可以是P个3D地图描述子各自的压缩数据的部分或全部数据进行第k级解压缩得到的。
在一些实施例中，第i级检索和第j级检索所采用的检索方式均为基于第二距离的检索方式，P个3D地图描述子属于S个3D地图描述子，P个3D地图描述子的各个3D地图描述子的第m重构数据的失真度，与P个3D地图描述子的各个3D地图描述子的第k重构数据的失真度不同，P个3D地图描述子的各个3D地图描述子的第m重构数据的失真度用于表示各个3D地图描述子的第m重构数据与相应的原始3D地图描述子之间的差别程度，P个3D地图描述子的各个3D地图描述子的第k重构数据的失真度用于表示各个3D地图描述子的第k重构数据与相应的原始3D地图描述子之间的差别程度，其中，P＜S。
在一些实施例中，P个3D地图描述子的各个3D地图描述子的第m重构数据的失真度，大于P个3D地图描述子的各个3D地图描述子的第k重构数据的失真度。至少两级检索中的后一级检索的结果相对于前一级检索的结果，失真度越来越小，从而提高了检索结果准确性。
举例而言，P个3D地图描述子属于S个3D地图描述子，以一个P个3D地图描述子中的一个3D地图描述子，m＝1，k＝2为例，该3D地图描述子的压缩数据包括第一量化数据和第二量化数据，该3D地图描述子的第一重构数据是对第一量化数据进行第一级解压缩得到，该3D地图描述子的第一重构数据可以包括该3D地图描述子的原始向量的部分分量的重构数据，该3D地图描述子的第二重构数据是对第二量化数据进行第二级解压缩得到，该3D地图描述子的第二重构数据可以包括该3D地图描述子的原始向量的另一部分分量的重构数据，即该3D地图描述子的压缩数据的解压缩过程包括第一(m＝1)级解压缩和第二(k＝2)级解压缩。
举例而言，P个3D地图描述子属于S个3D地图描述子，以一个P个3D地图描述子中的一个3D地图描述子，m＝1，k＝2为例，该3D地图描述子的压缩数据包括第一量化数据和第二量化数据，与上一举例不同，这里的第二量化数据可以是对第一量化数据的残差进行量化处理后得到的，该3D地图描述子的第一重构数据是对第一量化数据进行第一级解压缩得到，该3D地图描述子的第一重构数据可以包括该3D地图描述子的原始向量的第一重构数据(精度较低的重构数据)，该3D地图描述子的第二重构数据是对第二量化数据进行第二级解压缩得到，该3D地图描述子的第二重构数据可以包括该3D地图描述子的上述残差数据的重构数据，将该3D地图描述子的第一重构数据和该3D地图描述子的第二重构数据相加，得到该3D地图描述子的原始向量的重构数据(精度较高的重构数据)，即该3D地图描述子的压缩数据的解压缩过程包括第二(k＝2)级解压缩，第二(k＝2)级解压缩包括第一(m＝1)级解压缩。
S104C、根据检索描述子的部分数据或全部数据，在P个3D地图描述子的第二重构数据中进行第j级检索，以得到Q个3D地图描述子。
以下以第二重构数据是第k重构数据为例进行解释说明。
根据检索描述子的部分数据或全部数据，在P个3D地图描述子的第k重构数据中进行第j级检索，以得到Q个3D地图描述子。
S、P、Q、i、j、k和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
其中，S104C的具体解释说明可以参见图5A所示实施例的S104A的解释说明，此处不再赘述。
当然可以理解的，在S104C之后可以执行一个条件判断步骤，在满足条件时，可以结束本实施例的检索，在不满足条件时，可以继续执行一级或多级检索。这个条件可以是任意条件，其可以根据需求进行合理设置，例如，条件可以是相似度或相关度高于或等于预设阈值，也可以是检索得到的3D地图描述子的个数小于或等于预设个数等，本申请实施例不一一举例说明。
本申请实施例的3D地图的检索方法是一个片段流程，在该片段流程之前和/或之后还可以包括其他级检索，换言之，3D地图的检索方法可以包括比图5C所示的更多级数的多级检索，例如，3级、4级或5级等等。例如，如图5C所示实施例的S101C之前还可以有更多级的检索，或者，如图5C所示实施例的S104C之后还可以有更多级的检索，此处不一一举例说明。
在一些实施例中，通过多级检索得到的Q个3D地图描述子可以用于定位。若Q个3D地图描述子中的N个3D地图描述子对应的3D地图点与检索描述子对应的3D地图点匹配，则可以根据N个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿信息。该电子设备可以是采集上述视觉信息的电子设备。0＜N≤Q。
本实施例，对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的第一重构数据，根据检索描述子的部分数据或全部数据，在S个3D地图描述子的第一重构数据中进行第i级检索，筛选出P个3D地图描述子。对P个3D地图描述子的压缩数据进行第k级解压缩，得到P个3D地图描述子的第二重构数据，根据检索描述子的部分数据或全部数据，在P个3D地图描述子的第二重构数据中进行第j级检索，以筛选出更少个数的3D地图描述子。通过解压缩得到一级检索的3D地图描述子的一种重构数据，与一级检索交替进行。至少两级检索中存在一级检索为阶段性解压缩，可以提升检索速度，至少两级检索的解压缩差异化设置，可以保证检索准确率。任意两级检索各自所使用的3D地图描述子的一种重构数据的解压缩程度不同或失真度不同。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
参见图5D，图5D是本申请实施例提供的一种3D地图的检索方法的片段流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也可以应用于图1至图4f任一所示例的服务器。该方法包括但不限于以下步骤：
S101D、对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的重构数据，S个3D地图描述子对应3D地图中的多个3D地图点。
该重构数据可以指代第一重构数据、第二重构数据、……第m重构数据等任意一个，m为正整数。应当理解的是，第一重构数据、第二重构数据和第三重构数据等中的“第一”、“第二”和“第三”没有先后顺序之分，仅为了区分彼此而已。以下以重构数据是第m重构数据为例进行解释说明。
对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的第一重构数据，S个3D地图描述子对应3D地图中的多个3D地图点。
S102D、根据检索描述子的部分数据或全部数据，在S个3D地图描述子的重构数据中进行第i级检索，以得到P个3D地图描述子。
其中，S101D和S102D的解释说明可以参见图5C的S101C和S102C的解释说明，此处不再赘述。
S103D、从P个3D地图描述子的压缩数据中提取P个3D地图描述子的二值化数据。
S104D、根据检索描述子的二值化数据，在P个3D地图描述子的二值化数据中进行第j级检索，以得到Q个3D地图描述子。
S、P、Q、i、m和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数
其中，S103D和S104D的解释说明可以参见图5B的S103B和S104B的解释说明，此处不再赘述。
本实施例，对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述 子的重构数据，根据检索描述子的部分数据或全部数据，在S个3D地图描述子的重构数据中进行第i级检索，筛选出P个3D地图描述子。从P个3D地图描述子的压缩数据中提取所述P个描述子的二值化数据，根据检索描述子的二值化数据，在P个3D地图描述子的二值化数据中进行第j级检索，以筛选出更少个数的3D地图描述子。将从压缩数据提取或解压缩，和检索交替进行，以在3D地图中进行多级检索，得到可以用于定位的3D地图描述子。至少两级检索中存在一级检索为解压缩，可以提升检索准确性，至少两级检索中存在一级检索为从压缩数据中提取，可以提升检索速度。相较于在完全解压得到的3D地图的重构数据中进行检索，本申请实施例提供的3D地图的检索方法可以提升检索速度，并可以保证检索准确率。
需要说明的是，上述图5A至图5C所示的检索方法流程可以相互灵活组合，以形成各种不同的多级检索方法。
参见图6，图6是本申请实施例提供的一种3D地图的检索方法的流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也可以应用于图1至图4f任一所示例的服务器。本实施例以多级检索为4级检索为例进行举例说明，其中，第一级检索和第二级检索采集基于第一距离的检索方式，第三级检索和第四级检索采用基于第二距离的检索方式。该方法包括但不限于以下步骤：
开始执行本申请实施例的下述方法步骤。
S201、获取检索描述子。该检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征。
其中，S201的解释说明可以参见图5A所示实施例的S102A的解释说明，此处不再赘述。
S202、从S 1个3D地图描述子的压缩数据中提取S 1个3D地图描述子的第一(n＝1)二值化数据，并获取检索描述子的第一(n＝1)二值化数据。
S 1个3D地图描述子可以是S 1个区域描述子，或者S 1个3D地图点描述子。其中，一个区域描述子对应多个3D地图点，其具体解释说明可以参见前述实施例的解释说明，此处不再赘述。
从S 1个3D地图描述子的压缩数据的解释说明可以参见图5A至图5D所示实施例，此处不再赘述。
检索描述子的第一二值化数据可以是检索描述子的部分二值化数据。一种示例，本实施例的执行主体可以对检索描述子进行二值化处理，得到检索描述子的二值化数据，从检索描述子的二值化数据中选取部分作为检索描述子的第一二值化数据。
S203、根据检索描述子的第一(n＝1)二值化数据，在S 1个3D地图描述子的第一二值化数据中进行第一(i＝1)级检索，以得到P 1个3D地图描述子。
本实施例的执行主体可以使用检索描述子的第一二值化数据，在S 1个3D地图描述子的第一(n＝1)二值化数据中进行基于第一距离的检索，以得到与检索描述子的第一二值化数据最相似或最相关的P 1个3D地图描述子。P 1小于S 1。也可以理解为，通过检索得到一个子集，这个子集中的3D地图描述子的个数少于检索前的3D地图描述子集合中的3D地图描述子的个数。
S204、从P 1个3D地图描述子的压缩数据中提取P 1个3D地图描述子的第二(n＝2)二值化数据，并获取检索描述子的第二(n＝2)二值化数据。
检索描述子的第二(n＝2)二值化数据可以是检索描述子的部分或全部二值化数据。检索描述子的第二(n＝2)二值化数据与检索描述子的第一(n＝1)二值化数据的比特数不同。例如，检索描述子的第二(n＝2)二值化数据的比特数大于检索描述子的第一(n＝1)二值化数据的比特数。本实施例的第二二值化数据的比特开销可以大于第一二值化数据的比特开销，这样可以提升前一级检索的检索速度，并且可以保障后一级检索的准确率。
S205、根据检索描述子的第二(n＝2)二值化数据，在P 1个3D地图描述子的第二(n＝2)二值化数据中进行第二(j＝2)级检索，以得到Q 1个3D地图描述子。
本实施例的执行主体可以使用检索描述子的第二二值化数据，在P 1个3D地图描述子的第二二值化数据中进行基于第一距离的检索，以得到与检索描述子的第二二值化数据最相似或最相关的Q 1个D地图描述子。Q 1小于或等于P 1。也可以理解为，通过检索得到另一个子集，这个子集中的3D地图描述子的个数少于步骤203检索得到的3D地图描述子集合中的3D地图描述子的个数。
S206、对Q 1个3D地图描述子的压缩数据进行第一(m＝1)级解压缩，得到Q 1个3D地图描述子的第一(m＝1)重构数据。
S207、根据检索描述子的部分数据或全部数据，在Q 1个3D地图描述子的第一(m＝1)重构数据中进行第三级检索，以得到Q 2个3D地图描述子。
一种示例，本实施例的执行主体可以使用检索描述子的部分数据或全部数据，在Q 1个描述子的第一(m＝1)重构数据中进行基于第二距离的检索，以得到与检索描述子的部分数据或全部数据最相似或最相关的Q 2个3D地图描述子。Q 2小于或等于Q 1。也可以理解为，通过检索得到另一个子集，这个子集中的3D地图描述子的个数少于步骤205检索得到的3D地图描述子集合中的3D地图描述子的个数。
相较于前两级基于第一距离的检索，第三级采用基于第二距离的检索可以提升检索准确率。
S208、对Q 2个3D地图描述子的第二(k＝2)级解压缩，得到Q 2个3D地图描述子的第二(k＝2)重构数据。
一种示例，3D地图描述子的第一(m＝1)重构数据和第二(k＝2)重构数据的失真度不同。本实施例的3D地图描述子的第一(m＝1)重构数据的失真度高于第二(k＝2)重构数据的失真度，这样可以保障后一级检索的准确率。
S209、根据检索描述子的全部数据，在Q 2个3D地图描述子的第二(k＝2)重构数据中进行第四级检索，以得到Q 3个3D地图描述子。
一种示例，本实施例的执行主体可以使用检索描述子的全部数据，在Q 2个3D地图描述子的第二(k＝2)重构数据中进行检索，以得到与检索描述子的全部数据最相似或最相关的Q 3个3D地图描述子。Q 3小于或等于Q 2。也可以理解为，通过检索得到另一个子集，这个子集中的3D地图描述子的个数少于步骤207检索得到的3D地图描述子集合中的3D地图描述子的个数。
S210、根据Q 3个3D地图描述子对应的3D地图点进行定位，以得到电子设备的位姿 信息。
本实施例，通过从压缩数据中提取3D地图描述子的二值化数据或解压缩得到3D地图描述子的重构数据，与一级检索交替进行，以在3D地图中进行多级检索，得到可以用于定位的3D地图描述子。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
参见图7，图7是本申请实施例提供的一种3D地图的检索方法的流程示意图，在一些实现中，该方法可应用于图1至图4f任一所示例的电子设备，或者也可以应用于图1至图4f任一所示例的服务器。本实施例以图5A所示实施例的S个3D地图描述子为S个代表3D地图描述子进行举例说明。S个代表3D地图描述子各自分别对应至少一个数据集合。上述图5A的102A的具体可以方式可以包括但不限于以下步骤：
S1021、根据检索描述子的二值化数据，在S个代表3D地图描述子的二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子。
至少一个代表3D地图描述子对应至少一个数据集合，至少一个数据集合各自包括多个3D地图描述子的压缩数据。每个数据集合中的3D地图描述子具有一定相关性或相似性。
S个代表3D地图描述子的二值化数据，检索描述子的二值化数据可以是检索描述子的部分或全部二值化数据。本实施例的执行主体可以使用检索描述子的二值化数据，在S个代表3D地图描述子的二值化数据中进行基于第一距离的检索，以得到与检索描述子的二值化数据最相似或最相关的至少一个代表3D地图描述子。至少一个代表3D地图描述子的个数小于或等于S。
S1022、将至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为P个3D地图描述子。
通过检索得到的至少一个代表3D地图描述子，得到至少一个数据集合，从而得到至少一个数据集合中的P个3D地图描述子的压缩数据。
举例而言，S个代表3D地图描述子为T 1、T 2和T 3，T 1对应数据集合1，T 2对应数据集合2，T 3对应数据集合3。通过检索得到的至少一个代表3D地图描述子为T 3，从而得到数据集合3，数据集合3中可以包括P个3D地图描述子的压缩数据。
本实施例，通过在较少数量的代表3D地图描述子进行一级检索，以提升检索速度，之后可以在代表3D地图描述子对应的数据集合中进行一级或多级检索，以提升检索准确率。相较于在完全解压得到的3D地图的重构数据中进行检索，可以提升检索速度，并可以保证检索准确率。
需要说明的是图5C至图5D所示实施例的S个3D地图描述子为S个代表3D地图描述子的具体实施方式，与上述图7所示实施例类似。上述图5B的102B，或者图5C的102C，或者图5D的102D的具体可以方式可以参照图7所示实施例，此处不再赘述。
参见图8和图9，图8是本申请实施例提供的一种3D地图的检索方法的流程示意图。图9是本申请实施例提供的一种3D地图的检索方法的处理过程示意图。本实施例包括服 务器和电子设备。该方法包括但不限于以下步骤：
S401、服务器根据S个3D地图描述子的各个分量和预设阈值向量的对应分量，确定S个3D地图描述子的各个分量分别与预设阈值向量的对应分量之间的大小关系。
S402、服务器对上述大小关系进行二值化处理，得到S个3D地图描述子各自的二值化数据。
S403、服务器对S个3D地图描述子的各个分量分别与预设阈值向量的对应分量之间差值的绝对值进行量化处理，得到S个3D地图描述子各自的量化数据。
其中，预设阈值向量所包括的各个分量为任意数值。
S404、服务器对S个3D地图描述子各自的二值化数据和量化数据进行封装，得到3D地图的码流。
S405、服务器向电子设备发送3D地图的码流。
S406、电子设备获取检索描述子。
参照图9所示，电子设备可以从传感器所采集的视觉信息中提取检索描述子。
S407、电子设备对3D地图的码流进行解封装，得到S个3D地图描述子各自的二值化数据和量化数据，从中提取S个3D地图描述子各自的二值化数据。
参照图9所示，以S个3D地图描述子为5个3D地图描述子(T 1、T 2、T 3、T 4和T 5)为例，电子设备可以对3D地图的码流进行解封装，得到如图9所示的5个3D地图描述子各自的二值化数据和量化数据。
S408、电子设备根据检索描述子的二值化数据，在S个3D地图描述子各自的二值化数据中进行第一级检索，以得到P个3D地图描述子。
检索描述子的二值化数据通过如下方式获取：电子设备可以根据检索描述子的各个分量和上述预设阈值向量的对应分量，确定检索描述子的各个分量与预设阈值向量的对应分量之间的大小关系。对该大小关系进行二值化处理，得到检索描述子的二值化数据。在S个3D地图描述子各自的二值化数据中进行第一级检索可以采用基于第一距离的检索方式。
以图9做进一步举例说明，根据检索描述子的二值化数据，在5个3D地图描述子(T 1、T 2、T 3、T 4和T 5)中进行第一级检索。例如，可以分别计算5个3D地图描述子(T 1、T 2、T 3、T 4和T 5)各自的二值化数据与检索描述子的二值化数据之间的汉明距离，根据汉明距离筛选出3D地图描述子。这里以筛选出2个3D地图描述子为例，即筛选出如图9所示虚线框的T 1和T 2。也即P＝2。
S409、电子设备对P个3D地图描述子各自的量化数据进行解压缩，得到P个3D地图描述子各自的重构数据。
电子设备可以对P个3D地图描述子各自的量化数据进行反量化处理，得到P个3D地图描述子各自的反量化数据。根据P个3D地图描述子各自的反量化数据和二值化数据，得到P个3D地图描述子各自的重构数据。
S410、电子设备根据检索描述子，在P个3D地图描述子各自的重构数据中进行第二级检索，以得到Q个3D地图描述子。
在P个3D地图描述子各自的重构数据中进行第二级检索可以采集基于第二距离的 检索方式。
以图9做进一步举例说明，根据检索描述子的全部分量，在T 1和T 2中进行第二级检索。例如，可以分别计算T 1和T 2各自的重构数据与检索描述子之间的欧式距离，根据欧式距离筛选出3D地图描述子。这里以筛选出1个3D地图描述子为例，即筛选出如图9所示的T 1。也即Q＝1。
本实施例，通过服务器对3D地图的S个3D地图描述子进行压缩，得到S个3D地图描述子各自的二值化数据和量化数据，以减少传输3D地图所需的资源开销。通过电子设备获取S个3D地图描述子的二值化数据，根据检索描述子的二值化数据和S个3D地图描述子的二值化数据，采集基于第一距离的检索方式，快速筛选得到P个3D地图描述子，以提升检索速度。通过电子设备获取P个3D地图描述子的重构数据，根据检索描述子和P个3D地图描述子的重构数据，采集基于第二距离的检索方式，精准筛选得到Q个3D地图描述子，以提升检索精度。
上文结合附图对本申请实施例的3D地图的检索方法进行了详细的介绍，下面结合图10和图11对本申请实施例的3D地图的检索装置进行介绍。应理解，3D地图的检索装置能够执行本申请实施例的3D地图的检索方法。为了避免不必要的重复，下面在介绍本申请实施例的3D地图的检索装置时适当省略重复的描述。
参见图10，图10为本申请实施例提供的一种3D地图的检索装置的结构示意图。如图10所示，该3D地图的检索装置1000可以包括：检索模块1001、提取模块1002和解压缩模块1003。
在第一种可能的实现方式中，提取模块1002，用于从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块1001，用于根据检索描述子的二值化数据，在所述S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；解压缩模块1003，用于对所述P个3D地图描述子的压缩数据进行第m级解压缩，得到所述P个3D地图描述子的重构数据；所述P个3D地图描述子的压缩数据的解压缩过程至少包括所述第m级解压缩；所述检索模块1001，用于根据所述检索描述子的部分数据或全部数据，在所述P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
在一些实施例中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
在一些实施例中，所述第i级检索所采用的检索方式为基于第一距离的检索方式，所述第j级检索所采用的检索方式为基于第二距离的检索方式。
在一些实施例中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，接收所述视 觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
在一些实施例中，在N<Q的情况下，所述解压缩模块1003还用于：对所述Q个3D地图描述子的压缩数据进行第k级解压缩，以得到Q个3D地图描述子的重构数据；根据所述检索描述子的部分数据或全部数据，在所述Q个3D地图描述子的重构数据中进行第r级检索，以得到所述N个3D地图描述子；其中，N<Q<P，所述P个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，r和k为正整数，m<k，j<r≤L。
在一些实施例中，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压模块1003具体用于：对所述P个3D地图描述子各自的量化数据进行反量化处理，得到P个反量化数据，所述P个反量化数据用作所述P个3D地图描述子的重构数据；对所述Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据；根据所述Q个反量化数据和所述Q个3D地图描述子各自的二值化数据，得到所述Q个3D地图描述子各自的重构数据。
在一些实施例中，在N＝Q的情况下，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压缩模块1003，具体用于：对所述P个3D地图描述子各自的量化数据，进行反量化处理，得到P个反量化数据；根据所述P个反量化数据和所述P个3D地图描述子各自的二值化数据，得到所述P个3D地图描述子各自的重构数据。
在一些实施例中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块1001用于：根据所述检索描述子的二值化数据，在所述S个代表3D地图描述子的二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
在第二种可能的实现方式中，提取模块1002，用于从S个3D地图描述子的压缩数据中提取所述S个3D地图描述子的第一二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块1001，用于根据检索描述子的第一二值化数据，在所述S个3D地图描述子的第一二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；所述提取模块1002，还用于从所述P个3D地图描述子的压缩数据中提取所述P个3D地图描述子的第二二值化数据；所述检索模块1001，用于根据所述检索描述子的第二二值化数据，在所述P个3D地图描述子的第二二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
在一些实施例中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
在一些实施例中，所述第i级检索和所述第j级检索所采用的检索方式均为基于第一距离的检索，所述P个3D地图描述子属于所述S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据在所述3D地图描述子的压缩数据中的位置，不同于所述3D地图描述子的第二二值化数据在所述3D地图描述子的压缩数据中的位置，其中，P＜S。
在一些实施例中，所述P个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，小于所述P个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
在一些实施例中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的第一二值化数据和第二二值化数据。
在一些实施例中，所述检索描述子的第一二值化数据的长度等于所述S个3D地图描述子的各个3D地图描述子的第一二值化数据的长度，和/或，所述检索描述子的第二二值化数据的长度等于述S个3D地图描述子的各个3D地图描述子的第二二值化数据的长度。
在一些实施例中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块1001具体用于：根据所述检索描述子的第一二值化数据，在所述S个代表3D地图描述子的第一二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
在第三种可能的实现方式中，解压缩模块1003，用于对S个3D地图描述子的压缩数据进行第m级解压缩，以得到S个3D地图描述子的第一重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块1001，用于根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的第一重构数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；所述解压缩模块1003，还用于对所述P个3D地图描述子的压缩数据进行第k级解压缩，得到所述P个3D地图描述子的第二重构数据，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，或者，所述S个3D地图描述子的压缩数据的解压缩过程包括所述第k级解压缩，所述第k级解压缩包括第m级解压缩；检索模块1001，用于根据所述检索描述子的部分或全部，在所述P个3D地图描述子的第二重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j、k和m为正整数，且0＜Q＜P，0＜P＜T， 0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
在一些实施例中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
在一些实施例中，所述第i级检索和所述第j级检索所采用的检索方式均为基于第二距离的检索方式，所述P个3D地图描述子属于所述S个3D地图描述子，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，与所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度不同，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度用于表示所述各个3D地图描述子的第一重构数据与相应的原始3D地图描述子之间的差别程度，所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度用于表示所述各个3D地图描述子的第二重构数据与相应的原始3D地图描述子之间的差别程度，其中，P＜S。
在一些实施例中，所述P个3D地图描述子的各个3D地图描述子的第一重构数据的失真度，大于所述P个3D地图描述子的各个3D地图描述子的第二重构数据的失真度。
在一些实施例中，所述装置还包括：获取模块，用于：接收所述检索描述子，并获取所述检索描述子的部分数据或全部数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并获取所述检索描述子的部分数据或全部数据。
在一些实施例中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块1001具体用于：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的第m重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
在第四种可能的实现方式中，解压缩模块1003，用于对S个3D地图描述子的压缩数据进行第m级解压缩，得到S个3D地图描述子的重构数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；检索模块1001，用于根据检索描述子的部分数据或全部数据，在所述S个3D地图描述子的重构数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；提取模块1002，用于从所述P个3D地图描述子的压缩数据中提取所述P个描述子的二值化数据；所述检索模块1001，还用于根据所述检索描述子的二值化数据，在所述P个3D地图描述子的二值化数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、m和j为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
在一些实施例中，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；所述位姿确定模块，用于根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
在一些实施例中，所述第i级检索所采用的检索方式为基于第二距离的检索方式，所述第j级检索所采用的检索方式为基于第一距离的检索方式。
在一些实施例中，所述装置还包括：获取模块，用于：接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
在一些实施例中，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块1001具体用于：根据所述检索描述子的部分数据或全部数据，在所述S个代表3D地图描述子的第m重构数据中进行第i级检索，以得到至少一个代表3D地图描述子；将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
需要说明的是，3D地图的检索装置1000可以执行图5A至图5D任一或图6或图7所示实施例的3D地图的检索装置方法，或者，执行图8所示实施例的S406-S410的相关内容，或者，执行图9所示实施例的相关内容。具体的实现原理和技术效果可以参考上述方法实施例的具体解释说明，此处不再赘述。
图11为用于本申请实施例的译码装置1100的一种实现方式的示意性框图。其中，译码装置1100可以包括处理器1101、存储器1102和总线系统1103。其中，处理器1101和存储器1102通过总线系统1103相连，该存储器1102用于存储指令，该处理器1101用于执行该存储器1102存储的指令，以执行本申请描述的各种3D地图的检索方法。为避免重复，这里不再详细描述。
在本申请实施例中，该处理器1101可以是中央处理单元(central processingunit，CPU)，该处理器1101还可以是其他通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该存储器1102可以包括ROM设备或者RAM设备。任何其他适宜类型的存储设备也可以用作存储器1102。存储器1102可以包括由处理器1101使用总线1103访问的代码和数据11021。存储器1102可以进一步包括操作系统11023和应用程序11022，该应用程序11022包括允许处理器1101执行本申请描述的3D地图的检索方法的至少一个程序。例如，应用程序11022可以包括应用1至N，其进一步包括执行在本申请描述的3D地图的检索方法的3D地图应用。
该总线系统1103除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总 线等。但是为了清楚说明起见，在图中将各种总线都标为总线系统1103。
可选的，译码装置1100还可以包括一个或多个输出设备，诸如显示器1104。在一个示例中，显示器1104可以是触感显示器，其将显示器与可操作地感测触摸输入的触感单元合并。显示器1104可以经由总线1103连接到处理器1101。
需要指出的是，译码装置1100可以执行本申请中的3D地图的检索方法。
本领域技术人员能够领会，结合本文公开描述的各种说明性逻辑框、模块和算法步骤所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件来实施，那么各种说明性逻辑框、模块、和步骤描述的功能可作为一或多个指令或代码在计算机可读媒体上存储或传输，且由基于硬件的处理单元执行。计算机可读媒体可包含计算机可读存储媒体，其对应于有形媒体，例如数据存储媒体，或包括任何促进将计算机程序从一处传送到另一处的媒体(例如，根据通信协议)的通信媒体。以此方式，计算机可读媒体大体上可对应于(1)非暂时性的有形计算机可读存储媒体，或(2)通信媒体，例如信号或载波。数据存储媒体可为可由一或多个计算机或一或多个处理器存取以检索用于实施本申请中描述的技术的指令、代码和/或数据结构的任何可用媒体。计算机程序产品可包含计算机可读媒体。
作为实例而非限制，此类计算机可读存储媒体可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来存储指令或数据结构的形式的所要程序代码并且可由计算机存取的任何其它媒体。并且，任何连接被恰当地称作计算机可读媒体。举例来说，如果使用同轴缆线、光纤缆线、双绞线、数字订户线(DSL)或例如红外线、无线电和微波等无线技术从网站、服务器或其它远程源传输指令，那么同轴缆线、光纤缆线、双绞线、DSL或例如红外线、无线电和微波等无线技术包含在媒体的定义中。但是，应理解，所述计算机可读存储媒体和数据存储媒体并不包括连接、载波、信号或其它暂时媒体，而是实际上针对于非暂时性有形存储媒体。如本文中所使用，磁盘和光盘包含压缩光盘(CD)、激光光盘、光学光盘、数字多功能光盘(DVD)和蓝光光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光以光学方式再现数据。以上各项的组合也应包含在计算机可读媒体的范围内。
可通过例如一或多个数字信号处理器(DSP)、通用微处理器、专用集成电路(ASIC)、现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指前述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文中所描述的各种说明性逻辑框、模块、和步骤所描述的功能可以提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或者并入在组合编解码器中。而且，所述技术可完全实施于一或多个电路或逻辑元件中。
本申请的技术可在各种各样的装置或设备中实施，包含无线手持机、集成电路(IC)或一组IC(例如，芯片组)。本申请中描述各种组件、模块或单元是为了强调用于执行所揭示的技术的装置的功能方面，但未必需要由不同硬件单元实现。实际上，如上文所描述，各种单元可结合合适的软件和/或固件组合在编码解码器硬件单元中，或者通过互操作硬件单元(包含如上文所描述的一或多个处理器)来提供。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (18)

  1. 一种3D地图的检索方法，其特征在于，所述方法包括：
    从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；
    根据检索描述子的二值化数据，在所述S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；
    对所述P个3D地图描述子的压缩数据进行第m级解压缩，得到所述P个3D地图描述子的重构数据；所述P个3D地图描述子的压缩数据的解压缩过程至少包括所述第m级解压缩；
    根据所述检索描述子的部分数据或全部数据，在所述P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
  2. 根据权利要求1所述的方法，其特征在于，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图点匹配，N为正整数，0＜N≤Q，所述方法还包括：
    根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第i级检索所采用的检索方式为基于第一距离的检索方式，所述第j级检索所采用的检索方式为基于第二距离的检索方式。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，所述方法还包括：
    接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，
    接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，
    响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
  5. 根据权利要求2所述的方法，其特征在于，在N<Q的情况下，所述方法还包括：
    对所述Q个3D地图描述子的压缩数据进行第k级解压缩，以得到Q个3D地图描述子的重构数据；
    根据所述检索描述子的部分数据或全部数据，在所述Q个3D地图描述子的重构数据中进行第r级检索，以得到所述N个3D地图描述子；
    其中，N<Q<P，所述P个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，r和k为正整数，m<k，j<r≤L。
  6. 根据权利要求5所述的方法，其特征在于，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，
    所述对所述P个3D地图描述子的压缩数据进行第m级解压缩得到所述P个3D地图描述子的重构数据，包括：
    对所述P个3D地图描述子各自的量化数据进行反量化处理，得到P个反量化数据，所述P个反量化数据用作所述P个3D地图描述子的重构数据；
    所述对所述Q个3D地图描述子的压缩数据进行第k级解压缩得到所述Q个3D地图描述子的重构数据，包括：
    对所述Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据；
    根据所述Q个反量化数据和所述Q个3D地图描述子各自的二值化数据，得到所述Q个3D地图描述子各自的重构数据。
  7. 根据权利要求2所述的方法，其特征在于，在N＝Q的情况下，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述对所述P个3D地图描述子的压缩数据进行第m级解压缩得到所述P个3D地图描述子的重构数据，包括：
    对所述P个3D地图描述子各自的量化数据，进行反量化处理，得到P个反量化数据；
    根据所述P个反量化数据和所述P个3D地图描述子各自的二值化数据，得到所述P个3D地图描述子各自的重构数据。
  8. 根据权利要求1至7任一项所述的方法，其特征在于，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述根据检索描述子的二值化数据，在所述S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子，包括：
    根据所述检索描述子的二值化数据，在所述S个代表3D地图描述子的二值化数据中进行第i级检索，以得到至少一个代表3D地图描述子；
    将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
  9. 一种3D地图的检索装置，其特征在于，所述装置包括：
    提取模块，用于从S个3D地图描述子的压缩数据中提取S个3D地图描述子的二值化数据，所述S个3D地图描述子对应3D地图中的多个3D地图点；
    检索模块，用于根据检索描述子的二值化数据，在所述S个3D地图描述子的二值化数据中进行第i级检索，以得到P个3D地图描述子；所述检索描述子是从电子设备的传感器采集的视觉信息所提取出对应于真实环境的特征；
    解压缩模块，用于对所述P个3D地图描述子的压缩数据进行第m级解压缩，得到所述P个3D地图描述子的重构数据；所述P个3D地图描述子的压缩数据的解压缩过程至少包括所述第m级解压缩；
    所述检索模块，用于根据所述检索描述子的部分数据或全部数据，在所述P个3D地图描述子的重构数据中进行第j级检索，以得到Q个3D地图描述子，S、P、Q、i、j和m为正整数，且0＜Q＜P，0＜P＜T，0＜S≤T，T表示所述3D地图中的3D地图描述子的总个数，j＝i+1，1≤i<L，1＜j≤L，L表示所述3D地图的检索的总级数或者检索级数阈值，L为大于1的正整数。
  10. 根据权利要求9所述的装置，其特征在于，所述Q个3D地图描述子中的N个3D地图描述子用于定位，所述N个3D地图描述子对应的3D地图点与所述检索描述子对应的3D地图 点匹配，N为正整数，0＜N≤Q，所述装置还包括：位姿确定模块；
    所述位姿确定模块，根据所述N个3D地图描述子对应的3D地图点进行定位，以得到所述电子设备的位姿信息。
  11. 根据权利要求9或10所述的装置，其特征在于，所述第i级检索所采用的检索方式为基于第一距离的检索方式，所述第j级检索所采用的检索方式为基于第二距离的检索方式。
  12. 根据权利要求9至11任一项所述的装置，其特征在于，所述装置还包括：获取模块，用于：
    接收所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，
    接收所述视觉信息，从所述视觉信息中提取所述检索描述子，并对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据；或者，
    响应于用户输入的视觉信息采集的操作，触发所述传感器对所述真实环境进行视觉信息采集，得到所述视觉信息，从所述视觉信息中提取所述检索描述子，对所述检索描述子进行二值化处理，以得到所述检索描述子的二值化数据。
  13. 根据权利要求10所述的装置，其特征在于，在N<Q的情况下，所述解压缩模块还用于：
    对所述Q个3D地图描述子的压缩数据进行第k级解压缩，以得到Q个3D地图描述子的重构数据；
    根据所述检索描述子的部分数据或全部数据，在所述Q个3D地图描述子的重构数据中进行第r级检索，以得到所述N个3D地图描述子；
    其中，N<Q<P，所述P个3D地图描述子的压缩数据的解压缩过程包括所述第m级解压缩和所述第k级解压缩，r和k为正整数，m<k，j<r≤L。
  14. 根据权利要求13所述的装置，其特征在于，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压模块具体用于：
    对所述P个3D地图描述子各自的量化数据进行反量化处理，得到P个反量化数据，所述P个反量化数据用作所述P个3D地图描述子的重构数据；
    对所述Q个3D地图描述子各自的量化数据，进行反量化处理，得到Q个反量化数据；
    根据所述Q个反量化数据和所述Q个3D地图描述子各自的二值化数据，得到所述Q个3D地图描述子各自的重构数据。
  15. 根据权利要求10所述的装置，其特征在于，在N＝Q的情况下，所述P个3D地图描述子的压缩数据包括P个3D地图描述子各自的二值化数据和量化数据，所述解压缩模块具体用于：
    对所述P个3D地图描述子各自的量化数据，进行反量化处理，得到P个反量化数据；
    根据所述P个反量化数据和所述P个3D地图描述子各自的二值化数据，得到所述P个3D地图描述子各自的重构数据。
  16. 根据权利要求9至15任一项所述的装置，其特征在于，所述S个3D地图描述子为S个代表3D地图描述子，所述S个代表3D地图描述子各自分别对应至少一个数据集合，所述至少一个数据集合的各个数据集合包括至少一个3D地图描述子，所述检索模块用于：
    根据所述检索描述子的二值化数据，在所述S个代表3D地图描述子的二值化数据中进 行第i级检索，以得到至少一个代表3D地图描述子；
    将所述至少一个代表3D地图描述子各自对应的数据集合中的3D地图描述子，作为所述P个3D地图描述子。
  17. 一种3D地图的检索装置，其特征在于，包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-8中任一项所述的方法。
  18. 一种计算机可读存储介质，其特征在于，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行权利要求1-8中任一项所述的方法。
PCT/CN2021/105624 2021-06-04 2021-07-09 3d地图的检索方法和装置 WO2022252346A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202180099018.XA CN117460972A (zh) 2021-06-04 2021-07-09 3d地图的检索方法和装置
EP21943703.5A EP4336223A1 (en) 2021-06-04 2021-07-09 3d map retrieval method and apparatus
KR1020247000191A KR20240017906A (ko) 2021-06-04 2021-07-09 3d 지도 검색 방법 및 장치
US18/526,846 US20240111793A1 (en) 2021-06-04 2023-12-01 Method and apparatus for retrieving 3d map

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CNPCT/CN2021/098483 2021-06-04
PCT/CN2021/098482 WO2022252234A1 (zh) 2021-06-04 2021-06-04 3d地图的编码装置和方法
CNPCT/CN2021/098482 2021-06-04
PCT/CN2021/098483 WO2022252235A1 (zh) 2021-06-04 2021-06-04 3d地图的解码装置和方法、及3d地图的编码码流

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/526,846 Continuation US20240111793A1 (en) 2021-06-04 2023-12-01 Method and apparatus for retrieving 3d map

Publications (1)

Publication Number Publication Date
WO2022252346A1 true WO2022252346A1 (zh) 2022-12-08

Family

ID=84322741

Family Applications (4)

Application Number Title Priority Date Filing Date
PCT/CN2021/104601 WO2022252337A1 (zh) 2021-06-04 2021-07-05 3d地图的编解码方法及装置
PCT/CN2021/105624 WO2022252346A1 (zh) 2021-06-04 2021-07-09 3d地图的检索方法和装置
PCT/CN2021/105623 WO2022252345A1 (zh) 2021-06-04 2021-07-09 3d地图的压缩、解压缩方法和装置
PCT/CN2021/105625 WO2022252347A1 (zh) 2021-06-04 2021-07-09 3d地图的检索方法和装置

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/104601 WO2022252337A1 (zh) 2021-06-04 2021-07-05 3d地图的编解码方法及装置

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/CN2021/105623 WO2022252345A1 (zh) 2021-06-04 2021-07-09 3d地图的压缩、解压缩方法和装置
PCT/CN2021/105625 WO2022252347A1 (zh) 2021-06-04 2021-07-09 3d地图的检索方法和装置

Country Status (5)

Country Link
US (3) US20240112372A1 (zh)
EP (3) EP4336223A1 (zh)
KR (3) KR20240013238A (zh)
CN (4) CN117413523A (zh)
WO (4) WO2022252337A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000285121A (ja) * 1999-03-30 2000-10-13 Sony Corp 地図検索装置、地図検索方法、地図検索制御プログラムを記録した記録媒体、ナビゲーション装置、ナビゲート方法及びナビゲート制御プログラムを記録した記録媒体
CN102168983A (zh) * 2010-02-12 2011-08-31 三星电子株式会社 用于发送和接收地图数据的方法和装置
CN103745498A (zh) * 2014-01-16 2014-04-23 中国科学院自动化研究所 一种基于图像的快速定位方法
US20150043828A1 (en) * 2013-08-08 2015-02-12 Stmicroelectronics Sa Method for searching for a similar image in an image database based on a reference image
CN111551186A (zh) * 2019-11-29 2020-08-18 福瑞泰克智能系统有限公司 一种车辆实时定位方法、系统及车辆

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100238066B1 (ko) * 1996-06-26 2000-01-15 윤종용 양자화활동도를 이용한 비트 발생량 제어방법 및 그 영상 부호화장치
KR100668714B1 (ko) * 2005-01-14 2007-01-16 한국전자통신연구원 효과적인 텍스처 매핑을 위한 3차원 메쉬 정보의 텍스처좌표 부호화 및 복호화 방법
CN102611892B (zh) * 2006-03-16 2014-10-08 华为技术有限公司 在编码过程中实现自适应量化的方法及装置
US20120306850A1 (en) * 2011-06-02 2012-12-06 Microsoft Corporation Distributed asynchronous localization and mapping for augmented reality
CN104794219A (zh) * 2015-04-28 2015-07-22 杭州电子科技大学 一种基于地理位置信息的场景检索方法
CN116883522A (zh) * 2015-12-14 2023-10-13 松下电器（美国）知识产权公司 三维数据编码方法、解码方法、编码装置、解码装置
KR20170082797A (ko) * 2016-01-07 2017-07-17 주식회사 케이티 컨텐츠 기반 영상 검색을 위한 특징 서술자 부호화 방법 및 장치
EP3407607A1 (en) * 2017-05-24 2018-11-28 Thomson Licensing Method and device for encoding and reconstructing a point cloud
CN107590814A (zh) * 2017-09-01 2018-01-16 上海斐讯数据通信技术有限公司 一种三维图像的二值化处理方法及系统
CN108182205A (zh) * 2017-12-13 2018-06-19 南京信息工程大学 一种基于sift的hash算法的图像快速检索方法
JP7273029B2 (ja) * 2018-04-10 2023-05-12 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
JP7323545B2 (ja) * 2018-10-09 2023-08-08 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置
WO2020143005A1 (zh) * 2019-01-10 2020-07-16 深圳市大疆创新科技有限公司 对三维数据点集进行编码或解码的方法和设备
CN112384950A (zh) * 2019-06-12 2021-02-19 浙江大学 点云的编解码方法及装置
US20210042958A1 (en) * 2019-08-09 2021-02-11 Facebook Technologies, Llc Localization and mapping utilizing visual odometry
CN112348885A (zh) * 2019-08-09 2021-02-09 华为技术有限公司 视觉特征库的构建方法、视觉定位方法、装置和存储介质
CN110706248B (zh) * 2019-08-20 2024-03-12 广东工业大学 一种基于slam的视觉感知建图方法及移动机器人
CN111639147B (zh) * 2020-04-24 2021-02-05 视辰信息科技(上海)有限公司 一种地图压缩方法、系统及计算机可读存储介质
CN111882590A (zh) * 2020-06-24 2020-11-03 广州万维创新科技有限公司 一种基于单张图片定位的ar场景应用方法
CN111982152B (zh) * 2020-07-31 2022-04-05 广州文远知行科技有限公司 点云地图量化方法、装置、计算机设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000285121A (ja) * 1999-03-30 2000-10-13 Sony Corp 地図検索装置、地図検索方法、地図検索制御プログラムを記録した記録媒体、ナビゲーション装置、ナビゲート方法及びナビゲート制御プログラムを記録した記録媒体
CN102168983A (zh) * 2010-02-12 2011-08-31 三星电子株式会社 用于发送和接收地图数据的方法和装置
US20150043828A1 (en) * 2013-08-08 2015-02-12 Stmicroelectronics Sa Method for searching for a similar image in an image database based on a reference image
CN103745498A (zh) * 2014-01-16 2014-04-23 中国科学院自动化研究所 一种基于图像的快速定位方法
CN111551186A (zh) * 2019-11-29 2020-08-18 福瑞泰克智能系统有限公司 一种车辆实时定位方法、系统及车辆

Also Published As

Publication number Publication date
CN117461056A (zh) 2024-01-26
WO2022252347A1 (zh) 2022-12-08
US20240112372A1 (en) 2024-04-04
EP4339881A1 (en) 2024-03-20
KR20240017918A (ko) 2024-02-08
KR20240013238A (ko) 2024-01-30
CN117413523A (zh) 2024-01-16
US20240111793A1 (en) 2024-04-04
EP4336223A1 (en) 2024-03-13
CN117397242A (zh) 2024-01-12
WO2022252337A1 (zh) 2022-12-08
US20240095265A1 (en) 2024-03-21
EP4336454A1 (en) 2024-03-13
WO2022252345A1 (zh) 2022-12-08
CN117460972A (zh) 2024-01-26
KR20240017906A (ko) 2024-02-08

Similar Documents

Publication Publication Date Title
US20200175054A1 (en) System and method for determining a location on multimedia content
WO2019000880A1 (zh) 实景导航方法、装置、设备及计算机可读存储介质
JP2012160047A (ja) 対応参照画像検索装置及び方法、コンテンツ重畳装置、システム、及び方法、並びにコンピュータプログラム
CN113822427A (zh) 一种模型训练的方法、图像匹配的方法、装置及存储介质
CN111832579A (zh) 地图兴趣点数据处理方法、装置、电子设备以及可读介质
Guan et al. On-device mobile visual location recognition by using panoramic images and compressed sensing based visual descriptors
WO2021088497A1 (zh) 虚拟物体显示方法、全局地图更新方法以及设备
WO2022252346A1 (zh) 3d地图的检索方法和装置
WO2023051383A1 (zh) 一种设备定位方法、设备及系统
WO2022253228A1 (zh) 3d地图的压缩、解压缩方法和装置
WO2022252238A1 (zh) 3d地图的压缩、解压缩方法和装置
CN114925680A (zh) 物流兴趣点信息生成方法、装置、设备和计算机可读介质
WO2022252237A1 (zh) 3d地图的编解码方法及装置
WO2022252234A1 (zh) 3d地图的编码装置和方法
CN114610938A (zh) 遥感图像检索方法、装置、电子设备和计算机可读介质
WO2022252236A1 (zh) 3d地图的编解码方法及装置
US20240119639A1 (en) Apparatus and method for decoding 3d map, and encoded bitstream of 3d map

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943703

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023574725

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2021943703

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021943703

Country of ref document: EP

Effective date: 20231208

ENP Entry into the national phase

Ref document number: 20247000191

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 1020247000191

Country of ref document: KR

NENP Non-entry into the national phase

Ref country code: DE