CN117671183A - Map data compression method, device, equipment and storage medium - Google Patents

Abstract

The application provides a map data compression method, a device, equipment and a storage medium, which relate to the technical field of computers, and by adopting a visual feature point map storage architecture managed in a description subset, the description subset is stored when a map is created, redundant descriptors in some key frames are deleted after the creation is finished, and the data volume of a visual feature point map is reduced on the premise of ensuring that the accuracy of the map is not lost, wherein the method comprises the following steps: acquiring map data of a to-be-compressed visual feature point map, and determining target feature points in a plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data; and deleting the description sub-data of the target feature points from the plurality of description sub-data to obtain compressed map data.

Description

Map data compression method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for compressing map data.

Background

The conventional visual feature point map mainly stores two elements, namely a key frame and a map point, wherein the key frame stores feature points and descriptors corresponding to the feature points, and the map point stores the descriptors of the map points and the association information of the descriptors and the feature points.

With the continuous expansion of the map scale, the number of map points is increased to millions or even higher, so that the file size of the map may be large, which is inconvenient for network transmission and storage. Map compression is one of the effective means for solving the above-described problems.

The current common map compression method is often focused on compressing or deleting map points in a three-dimensional map, so that the three-dimensional points of the three-dimensional map after compression are reduced, and the precision of the three-dimensional map is greatly influenced.

Disclosure of Invention

Based on the technical problems, the application provides a map data compression method, device, equipment and storage medium, which adopt a visual feature point map storage architecture managed in a description subset, store the description subset when a map is created, delete redundant descriptors in some key frames after the creation is finished, and reduce the data volume of a visual feature point map on the premise of ensuring that the map accuracy is not lost.

In a first aspect, the present application provides a map data compression method, the method comprising: acquiring map data of a to-be-compressed visual feature point map; the map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point; determining target feature points in the plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data; and deleting the description sub-data of the target feature points from the plurality of description sub-data to obtain compressed map data.

The map data of the to-be-compressed visual feature point map comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises the description sub-data of each feature point and the description sub-data of each three-dimensional map point, so that the key frame and description sub-sets contained in the map points can be managed conveniently, and further the follow-up operation of deleting the description meeting the requirement of addition can be executed. Further, the present application determines that feature points (i.e., target feature points) of corresponding three-dimensional map points do not exist in the plurality of key frame data, and deletes descriptor data of the feature points, so as to reduce the number of descriptor data. In this way, the redundant deletion of descriptors is realized by adopting the method for centrally managing the descriptors contained in the existing key frames and map points, so that the three-dimensional map points and key words in the map are reserved while the data volume of the map data is reduced, and the influence on the map precision is low. In addition, the to-be-compressed visual feature point map pair description subset is stored, and loading and operation speeds are higher when two description sub-distances are calculated.

In one possible implementation manner, one key frame data corresponds to one key frame data structure, a plurality of key frame data corresponds to a plurality of continuously stored key frame data structures, one three-dimensional map point data corresponds to one three-dimensional map point data structure, a plurality of three-dimensional map point data corresponds to a plurality of continuously stored three-dimensional map point data structures, one descriptor data corresponds to one descriptor data structure, and a plurality of descriptor data corresponds to a plurality of continuously stored descriptor data structures; obtaining map data of a visual feature point map to be compressed, including: creating a key frame pointer array, a three-dimensional map point pointer array and a description sub-array head address variable; the key frame pointer array is used for storing pointers of each key frame data structure in the visual characteristic point map data to be compressed, the three-dimensional map point pointer array is used for storing pointers of each three-dimensional map point data structure in the visual characteristic point map data to be compressed, and the description sub-array head address variable is used for storing sequence head addresses of a plurality of continuous description sub-data structures; and taking the key frame pointer array, the three-dimensional map point pointer array and the description sub-array head address variable as map handles to load map data of the visual feature point map to be compressed.

In a possible implementation manner, after deleting the descriptor data of the target feature point, the method further includes: creating an idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data; for any traversed current feature point, determining a descriptor pointer corresponding to the descriptor data of the current feature point, and adding the descriptor pointer to an idle descriptor pointer queue; judging whether the current characteristic points have corresponding three-dimensional map points or not; if the corresponding three-dimensional map points exist in the current feature points, a pointer is obtained from the idle description sub-pointer queue, the description sub-data of the current feature points are copied to the memory addresses corresponding to the pointers, and the pointers are deleted from the idle description sub-pointer queue.

In one possible implementation, a plurality of reserved spaces are reserved in front of a plurality of continuously stored descriptor data structures, and one reserved space can accommodate one descriptor data; after deleting the descriptor data of the target feature point, the method further comprises: creating an idle description sub-pointer queue, and adding pointers of a plurality of reserved spaces to the idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data, and determining a description sub pointer corresponding to the description sub data of each feature point; judging whether each characteristic point has a corresponding three-dimensional map point or not; if one feature point does not have a corresponding three-dimensional map point, adding a descriptor pointer corresponding to descriptor data of the feature point to an idle descriptor pointer queue; if one feature point has a corresponding three-dimensional map point, acquiring a pointer from an idle descriptor pointer queue, and adding an original descriptor pointer corresponding to descriptor data of the feature point to the idle descriptor pointer queue after copying the descriptor data of the feature point to a memory address corresponding to the pointer; the memory address is any one reserved space or any memory address corresponding to the descriptor pointer added to the idle pointer queue.

In one possible implementation, a pointer is obtained from a free descriptor pointer queue, including: and acquiring the pointer which enters the idle description sub-pointer queue earliest according to the time when each pointer in the idle description sub-pointer queue enters the idle description sub-pointer queue.

In one possible implementation manner, the key frame data further includes a descriptor sequence number corresponding to each feature point in the key frame, one feature point corresponds to one descriptor, and the descriptor sequence number is used for reflecting the position of the descriptor corresponding to one feature point in the plurality of descriptor data; after loading map data of a to-be-compressed visual feature point map, determining a description sub pointer corresponding to the description sub data of each feature point, wherein the method comprises the following steps: for any one feature point, a descriptor sequence number corresponding to the feature point is obtained; determining the position offset between the descriptor and the first address of the descriptor data structure in the descriptor array first address variable according to the size of one description data in the visual feature point map data to be compressed and the descriptor sequence number corresponding to the feature point; and determining a descriptor pointer corresponding to the descriptor data of the feature point based on the first address and the position bias of the descriptor data structure in the descriptor array first address variable.

In a possible implementation manner, the method further includes: traversing each three-dimensional map point data, and determining a description sub pointer corresponding to the description sub data of each three-dimensional map point data; for the description sub-data of any three-dimensional map point data, a pointer is obtained from an idle description sub-pointer queue, the description sub-data of the three-dimensional map point data is copied to a storage space corresponding to the pointer, and an original description sub-pointer corresponding to the description sub-data of the three-dimensional map point data is added to the idle description sub-pointer queue so that the original description sub-pointer can be used by a subsequent description sub-data structure.

In a possible implementation manner, after copying the descriptor data of the feature point into the reserved space corresponding to the pointer, the method further includes: based on the current pointer of the descriptor data of the feature points and the first address of the descriptor data structure in the descriptor array first address variable, the current descriptor sequence number corresponding to the feature points is determined, and the original descriptor sequence number corresponding to the feature points in the key frame data is updated based on the current descriptor sequence number.

In a possible implementation manner, one three-dimensional map point corresponds to one descriptor data, one three-dimensional map point data further comprises a descriptor sequence number corresponding to the three-dimensional map point, and the descriptor sequence number is used for reflecting the position of the descriptor data corresponding to the three-dimensional map point in the plurality of descriptor data; after copying the description sub-data of the three-dimensional map point data to the reserved space corresponding to the pointer, the method further comprises the following steps: based on the current pointer of the description sub data of the three-dimensional map point data and the first address of the description sub data structure in the description sub array first address variable, determining the current description sub sequence number corresponding to the three-dimensional map point, and based on the current description sub sequence number, updating the original description sub sequence number corresponding to the three-dimensional map point in the three-dimensional map point data.

In a possible implementation manner, after copying the descriptor data meeting the condition to the memory address corresponding to the idle descriptor pointer queue pointer, the method further includes: determining the position of a valid description sub-data structure in a plurality of continuously stored description sub-data structures; the descriptor data structure is not empty to be a valid descriptor data structure; discarding storage space after the effective description sub-data structure in the plurality of continuously stored description sub-data structures to obtain a new plurality of continuously stored description sub-data structures; the number of new plurality of consecutively stored description sub-data structures is less than the number of plurality of consecutively stored description sub-data structures.

In a possible implementation manner, the method further includes: responding to a key frame adding request, and acquiring an idle key frame pointer from an idle key frame pointer queue; the key frame adding request is used for requesting to insert a new key frame into the visual feature point map to be compressed, and the idle key frame pointer queue is used for collecting memory addresses which can be used for moving the key frames; copying data except the description sub data in the new key frame to a memory address corresponding to the idle key frame pointer; and acquiring an idle description sub-pointer from the idle description sub-pointer queue, copying description sub-data in the new key frame to a memory address corresponding to the idle description sub-pointer, and storing the memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle key frame pointer.

In a possible implementation manner, the method further includes: responding to a map point adding request, and acquiring an idle map point pointer from an idle map point pointer queue; the map point adding request is used for requesting to insert new map points into the visual feature point map to be compressed, and the idle map point pointer queue is used for collecting memory addresses which can be used for moving the map points; copying data except the description sub data in the new map point to a memory address corresponding to the idle map point pointer; and acquiring an idle description sub pointer from the idle description sub pointer queue, copying description sub data in the new map point to a memory address corresponding to the idle description sub pointer, and storing the memory address information corresponding to the idle description sub pointer to the memory address corresponding to the idle map point pointer.

In a second aspect, the present application provides a map data compression apparatus, the apparatus including an acquisition unit, a determination unit, and a processing unit; the acquisition unit is used for acquiring map data of the visual feature point map to be compressed; the map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point; a determining unit configured to determine target feature points in the plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data; and the processing unit is used for deleting the description sub-data of the target feature points from the plurality of description sub-data to obtain compressed map data.

In one possible implementation manner, one key frame data corresponds to one key frame data structure, a plurality of key frame data corresponds to a plurality of continuously stored key frame data structures, one three-dimensional map point data corresponds to one three-dimensional map point data structure, a plurality of three-dimensional map point data corresponds to a plurality of continuously stored three-dimensional map point data structures, one descriptor data corresponds to one descriptor data structure, and a plurality of descriptor data corresponds to a plurality of continuously stored descriptor data structures; the acquisition unit is specifically configured to: creating a key frame pointer array, a three-dimensional map point pointer array and a description sub-array head address variable; the key frame pointer array is used for storing pointers of each key frame data structure in the visual characteristic point map data to be compressed, the three-dimensional map point pointer array is used for storing pointers of each three-dimensional map point data structure in the visual characteristic point map data to be compressed, and the description sub-array head address variable is used for storing sequence head addresses of a plurality of continuous description sub-data structures; and taking the key frame pointer array, the three-dimensional map point pointer array and the description sub-array head address variable as map handles to load map data of the visual feature point map to be compressed.

In a possible implementation manner, after deleting the descriptor data of the target feature point, the processing unit is further configured to: creating an idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data; for any traversed current feature point, determining a descriptor pointer corresponding to the descriptor data of the current feature point, and adding the descriptor pointer to an idle descriptor pointer queue; judging whether the current characteristic points have corresponding three-dimensional map points or not; if the corresponding three-dimensional map points exist in the current feature points, a pointer is obtained from the idle description sub-pointer queue, the description sub-data of the current feature points are copied to the memory addresses corresponding to the pointers, and the pointers are deleted from the idle description sub-pointer queue.

In one possible implementation, a plurality of reserved spaces are reserved in front of a plurality of continuously stored descriptor data structures, and one reserved space can accommodate one descriptor data; after deleting the descriptor data of the target feature point, the processing unit is further configured to: creating an idle description sub-pointer queue, and adding pointers of a plurality of reserved spaces to the idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data, and determining a description sub pointer corresponding to the description sub data of each feature point; judging whether each characteristic point has a corresponding three-dimensional map point or not; if one feature point does not have a corresponding three-dimensional map point, adding a descriptor pointer corresponding to descriptor data of the feature point to an idle descriptor pointer queue; if one feature point has a corresponding three-dimensional map point, acquiring a pointer from an idle descriptor pointer queue, and adding an original descriptor pointer corresponding to descriptor data of the feature point to the idle descriptor pointer queue after copying the descriptor data of the feature point to a memory address corresponding to the pointer; the memory address is any one reserved space or any memory address corresponding to the descriptor pointer added to the idle pointer queue.

In a possible implementation manner, the processing unit is specifically configured to: and acquiring the pointer which enters the idle description sub-pointer queue earliest according to the time when each pointer in the idle description sub-pointer queue enters the idle description sub-pointer queue.

In one possible implementation manner, the key frame data further includes a descriptor sequence number corresponding to each feature point in the key frame, one feature point corresponds to one descriptor, and the descriptor sequence number is used for reflecting the position of the descriptor corresponding to one feature point in the plurality of descriptor data; after loading the map data of the visual feature point map to be compressed, the processing unit is specifically configured to: for any one feature point, a descriptor sequence number corresponding to the feature point is obtained; determining the position offset between the descriptor and the first address of the descriptor data structure in the descriptor array first address variable according to the size of one description data in the visual feature point map data to be compressed and the descriptor sequence number corresponding to the feature point; and determining a descriptor pointer corresponding to the descriptor data of the feature point based on the first address and the position bias of the descriptor data structure in the descriptor array first address variable.

In a possible implementation, the processing unit is further configured to: traversing each three-dimensional map point data, and determining a description sub pointer corresponding to the description sub data of each three-dimensional map point data; for the description sub-data of any three-dimensional map point data, a pointer is obtained from an idle description sub-pointer queue, the description sub-data of the three-dimensional map point data is copied to a storage space corresponding to the pointer, and an original description sub-pointer corresponding to the description sub-data of the three-dimensional map point data is added to the idle description sub-pointer queue so that the original description sub-pointer can be used by a subsequent description sub-data structure.

In a possible implementation manner, after copying the descriptor data of the feature point into the reserved space corresponding to the pointer, the processing unit is further configured to: based on the current pointer of the descriptor data of the feature points and the first address of the descriptor data structure in the descriptor array first address variable, the current descriptor sequence number corresponding to the feature points is determined, and the original descriptor sequence number corresponding to the feature points in the key frame data is updated based on the current descriptor sequence number.

In a possible implementation manner, one three-dimensional map point corresponds to one descriptor data, one three-dimensional map point data further comprises a descriptor sequence number corresponding to the three-dimensional map point, and the descriptor sequence number is used for reflecting the position of the descriptor data corresponding to the three-dimensional map point in the plurality of descriptor data; after copying the description sub-data of the three-dimensional map point data to the reserved space corresponding to the pointer, the processing unit is further configured to: based on the current pointer of the description sub data of the three-dimensional map point data and the first address of the description sub data structure in the description sub array first address variable, determining the current description sub sequence number corresponding to the three-dimensional map point, and based on the current description sub sequence number, updating the original description sub sequence number corresponding to the three-dimensional map point in the three-dimensional map point data.

In a possible implementation manner, after copying the descriptor data meeting the condition to the memory address corresponding to the idle descriptor pointer queue pointer, the processing unit is further configured to: determining the position of a valid description sub-data structure in a plurality of continuously stored description sub-data structures; the descriptor data structure is not empty to be a valid descriptor data structure; discarding storage space after the effective description sub-data structure in the plurality of continuously stored description sub-data structures to obtain a new plurality of continuously stored description sub-data structures; the number of new plurality of consecutively stored description sub-data structures is less than the number of plurality of consecutively stored description sub-data structures.

In a possible implementation, the processing unit is further configured to: responding to a key frame adding request, and acquiring an idle key frame pointer from an idle key frame pointer queue; the key frame adding request is used for requesting to insert a new key frame into the visual feature point map to be compressed, and the idle key frame pointer queue is used for collecting memory addresses which can be used for moving the key frames; copying data except the description sub data in the new key frame to a memory address corresponding to the idle key frame pointer; and acquiring an idle description sub-pointer from the idle description sub-pointer queue, copying description sub-data in the new key frame to a memory address corresponding to the idle description sub-pointer, and storing the memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle key frame pointer.

In a possible implementation, the processing unit is further configured to: responding to a map point adding request, and acquiring an idle map point pointer from an idle map point pointer queue; the map point adding request is used for requesting to insert new map points into the visual feature point map to be compressed, and the idle map point pointer queue is used for collecting memory addresses which can be used for moving the map points; copying data except the description sub data in the new map point to a memory address corresponding to the idle map point pointer; and acquiring an idle description sub pointer from the idle description sub pointer queue, copying description sub data in the new map point to a memory address corresponding to the idle description sub pointer, and storing the memory address information corresponding to the idle description sub pointer to the memory address corresponding to the idle map point pointer.

In a third aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory stores instructions executable by the processor; the processor is configured to execute the instructions to cause the electronic device to implement the method of the first aspect as described above.

In a fourth aspect, the present application provides a computer program product for, when run in an electronic device, causing the electronic device to perform the method of the first aspect described above, to carry out the method of the first aspect described above.

In a fifth aspect, the present application provides a computer readable storage medium comprising: a software instruction; the software instructions, when executed in an electronic device, cause the electronic device to implement the method of the first aspect described above.

The advantages of the second to fifth aspects described above may refer to the first aspect, and are not repeated here.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of a map data compression system according to an embodiment of the present application;

fig. 2 is a schematic diagram of the composition of an electronic device according to an embodiment of the present application;

fig. 3 is a schematic flow chart of a map data compression method according to an embodiment of the present application;

fig. 4 is a schematic diagram of a map data storage architecture according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a map handle according to an embodiment of the present disclosure;

FIG. 6 is a second flow chart of a map data compression method according to the embodiment of the present application;

FIG. 7 is a flowchart of pointer address calculation according to an embodiment of the present application;

FIG. 8 is a diagram of map compression before and after comparison according to an embodiment of the present application;

fig. 9 is a schematic diagram of the composition of the map data compression device according to the embodiment of the present application.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.

In addition, in the description of the embodiments of the present application, "/" means or, unless otherwise indicated, for example, a/B may mean a or B. "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

Before explaining the embodiments of the present application in detail, some related terms and related techniques related to the embodiments of the present application are described.

In the aspect of auxiliary driving, for realizing functions such as autonomous parking of a vehicle, a person often needs to manually drive the vehicle to teach and run, and an automatic driving system can continuously collect sensor information during teaching to establish a parking map. In the parking stage, after the vehicle runs to the starting point of the map, the automatic driving system can guide the vehicle to run according to the existing parking map and park in the appointed parking space.

In the map building process, the vision synchronous positioning and map building (Visual simultaneous localization and mapping, vsram) system can collect image information in the environment through the image acquisition device, infer the motion condition of the image acquisition device in the environment, realize the positioning of the image acquisition device, model the whole environment and build a global consistent map.

The vsram uses a visual sensor to construct a map by extracting feature points from an image and associating the feature points with descriptors of the feature points. Compared with the laser radar, millimeter wave radar, inertial measurement unit (Inertial Measurement Unit, IMU) and other sensors, the visual sensor contains more abundant information, so that visual information is often used for map construction in the auxiliary driving field to obtain a visual characteristic point map.

However, the classical visual feature point map mainly comprises two parts of a key frame and map points, wherein a large number of two-dimensional feature points and corresponding descriptive sub-information are stored in the key frame, and the key frame is used for searching and matching during parking positioning. However, the situation causes that the existing visual feature point mapping method often generates a larger map, and the vehicle-mounted scene is often limited by the performance and the power consumption of the platform, and a larger map space is difficult to reserve, so that the application of the visual feature point mapping method is influenced.

Map compression is one of the effective means for solving the problems, and the current common map compression method mainly aims at compressing map points in a map, and aims at deleting map points with low observation through the observation quantity of the map points to reduce the size of the map. However, the reduction of map points has a great influence on the accuracy of the three-dimensional map, and further reduces the use value of the compressed map.

In view of the above, the embodiments of the present application provide a map data compression method, which reduces the size of a map by compressing descriptors of feature points in a key frame and deleting descriptors corresponding to two-dimensional feature points that are not associated with three-dimensional map points, and reduces the data size of map data while reducing the loss of map precision due to the retention of three-dimensional map points in the map.

The map data compression method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

The information extraction method provided by the embodiment of the application can be applied to a map data compression system, and fig. 1 shows a schematic structural diagram of the map data compression system. As shown in fig. 1, the map data compression system 10 includes a map data compression device 11 and a user terminal 12. Wherein the map data compression device 11 is connected to the user terminal 12. The map data compression device 11 and the user terminal 12 may be connected by a wired manner or may be connected by a wireless manner, which is not limited in the embodiment of the present application.

After acquiring the map data of the visual feature point map to be compressed, the map data compression device 11 may compress the map data of the visual feature point map to be compressed to obtain compressed map data, and transmit the compressed map data to the user terminal 12. The specific compression process may refer to the map data compression method in the following method embodiments, which is not described herein.

Accordingly, the user terminal 12 may perform some positioning operations, etc., based on the received compressed map data.

The map data compression device 11 may be any electronic device having a data processing function, and for example, the map data compression device 11 may be a server, a computer, or a server cluster including a plurality of servers. In some implementations, the server cluster may also be a distributed cluster. Alternatively, the server may be a central server, and the server may also be implemented on a cloud platform, for example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud (community cloud), a distributed cloud, an inter-cloud (inter-cloud), a multi-cloud (multi-cloud), and the like, or any combination thereof. The embodiments of the present application are not limited in this regard.

The user terminal 12 may be a mobile phone, a vehicle-mounted terminal, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), a desktop computer, an edge server, etc., and the specific type of the user terminal 12 is not limited in the embodiment of the present application.

In fig. 1, the map data compression device 11 and the user terminal 12 are described as separate devices, and alternatively, the map data compression device 11 and the user terminal 12 may be combined into one device. For example, the map data compression device 11 or its corresponding function, and the user terminal 12 or its corresponding function may be integrated in one device. The embodiments of the present application are not limited in this regard.

The main execution subject of the map data compression method provided in the embodiment of the present application may be the map data compression device 11 described above. As described above, the map data compression device 11 may be an electronic device having a data processing function, such as a computer or a server. Alternatively, the map data compression device 11 may also be a processor (e.g., a central processing unit (central processing unit, CPU)) in the aforementioned electronic apparatus; alternatively, the map data compression device 11 may be an Application (APP) having a model training function installed in the electronic apparatus; alternatively, the map data compression device 11 may be a functional module or the like having a model training function in the electronic device. The embodiments of the present application are not limited in this regard.

For simplicity of description, the map data compression device 11 will be described below as an example of an electronic device.

Fig. 2 is a schematic diagram of the composition of an electronic device according to an embodiment of the present application. As shown in fig. 2, the electronic device may include: processor 20, memory 21, communication line 22, and communication interface 23, and input-output interface 24.

The processor 20, the memory 21, the communication interface 23, and the input/output interface 24 may be connected by a communication line 22.

The processor 20 is configured to execute instructions stored in the memory 21 to implement a fault analysis method provided in the following embodiments of the present application. The processor 20 may be a CPU, general purpose processor network processor (network processor, NP), digital signal processor (digital signal processing, DSP), microprocessor, microcontroller (micro control unit, MCU), programmable logic device (programmable logic device, PLD), or any combination thereof. The processor 20 may also be any other apparatus having a processing function, such as a circuit, a device, or a software module, which is not limited in this embodiment. In one example, processor 20 may include one or more CPUs, such as CPU0 and CPU1 in fig. 2. As an alternative implementation, the electronic device may include multiple processors, for example, processor 25 (illustrated in phantom in fig. 2) in addition to processor 20.

A memory 21 for storing instructions. For example, the instructions may be a computer program. Alternatively, the memory 21 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and/or instructions, an access memory (random access memory, RAM) or other type of dynamic storage device capable of storing information and/or instructions, or an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), a compact disc (compact disc read-only memory, CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage medium, or other magnetic storage device, etc., which are not limited in this embodiment.

It should be noted that, the memory 21 may exist separately from the processor 20 or may be integrated with the processor 20. The memory 21 may be located inside the electronic device or may be located outside the electronic device, which is not limited in the embodiment of the present application.

Communication lines 22 for conveying information between components included in the electronic device.

A communication interface 23 for communicating with other devices or other communication networks. The other communication network may be an ethernet, a radio access network (radio access network, RAN), a wireless local area network (wireless local area networks, WLAN), etc. The communication interface 23 may be a module, a circuit, a transceiver, or any device capable of enabling communication.

And an input-output interface 24 for enabling human-machine interaction between the user and the electronic device. Such as enabling action interactions or information interactions between a user and an electronic device.

The input/output interface 24 may be a mouse, a keyboard, a display screen, or a touch-sensitive display screen, for example. The action interaction or information interaction between the user and the electronic equipment can be realized through a mouse, a keyboard, a display screen, a touch display screen or the like.

It should be noted that the structure shown in fig. 2 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown in fig. 2, or a combination of some components, or a different arrangement of components.

The map data compression method provided in the embodiment of the present application is described below.

Fig. 3 is a flow chart of a map data compression method according to an embodiment of the present application. Alternatively, the method may be performed by an electronic apparatus having the above-described hardware structure shown in fig. 2, and as shown in fig. 3, the method includes S301 to S303.

S301, acquiring map data of a to-be-compressed visual feature point map.

The map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point.

It should be noted that a key frame is a representative frame among several frames of normal frames. Feature points are special places in the keyframe, for example, corner points, edges and blocks in the keyframe can be regarded as representative places in the keyframe. A three-dimensional map point (hereinafter referred to as map point) is a three-dimensional point, which is a three-dimensional object from the real world, and has a unique identification. The feature points in different key frames may correspond to the same three-dimensional map point in the three-dimensional space, but most of the feature points do not have corresponding three-dimensional map points in the three-dimensional space (for example, in practical application, a considerable number of feature points may not be successfully associated with corresponding three-dimensional map points due to factors such as shielding or scene change when the feature points are mapped, so that the corresponding three-dimensional map points do not exist in the three-dimensional space). A descriptor is a method used in computer vision to describe features of an image or object, and may be designed according to the principle that "features that are similar in appearance should have similar descriptors". The descriptors of the feature points may reflect information of pixels surrounding the feature points. The descriptors of the three-dimensional map points may reflect the local features of the three-dimensional object. The embodiment of the application does not limit the specific generation method of the descriptors of the feature points and the descriptors of the three-dimensional map points.

Alternatively, the embodiment of the application proposes a map data storage architecture, which may also be called a static map. The following variables are stored in the architecture, mainly in order: static map element attribute information, a key frame data structure, a three-dimensional map point data structure, and a description sub-data structure. The static map element attribute information stores some attribute information of the static map, such as check codes, map sizes, the number of stored key frames, the number of stored map points, the number of stored effective descriptors and the like. The key frame data structure may be an array of structures storing all key frame information in the static map, each key frame structure storing all two-dimensional feature points observed by the key frame, and a descriptor index corresponding to the two-dimensional feature points (recording which bit of the descriptor is stored in the descriptor array). The map point data structure may be a structure array, storing all map point information in the static map, and each map point stores a descriptor index corresponding to the map point (which bit of the descriptor is stored in the descriptor array is recorded). The description sub-data structure can be a structure body array, and stores the description sub-data of each feature point in all key frames in the static map and the description sub-data of all map points.

As shown in fig. 4, one key frame data corresponds to one key frame data structure, a plurality of key frame data corresponds to a plurality of continuously stored key frame data structures, M key frame data structures are continuously stored in the electronic device, one three-dimensional map point data corresponds to one three-dimensional map point data structure, a plurality of three-dimensional map point data corresponds to a plurality of continuously stored three-dimensional map point data structures, N map point data structures are continuously stored in the electronic device, one descriptor data corresponds to one descriptor data structure, a plurality of descriptor data corresponds to a plurality of continuously stored descriptor data structures, and K descriptor data structures are continuously stored in the electronic device.

It can be understood that the data of different types are closely arranged when the static map is stored and do not contain any pointer information, so that the static map is not required to be modified when being loaded, the corresponding data can be directly extracted, and the loading speed is greatly improved. In addition, through the map architecture with the separated features, the descriptor storage is decoupled from the key frame and map point storage, so that the subsequent map compression is convenient.

As one possible implementation, the electronic device may pre-construct a map handle and access static map data stored in the electronic device hard disk via the map handle to successfully load the visual feature point map to be compressed.

Alternatively, as shown in FIG. 5, the map handle portion stores mainly the following variables for interacting with the static map. This part of the information electronic device is not stored when the map is saved, but is constructed when the map is used. The key frame pointer array is used for sequentially storing pointers of each key frame structure body in the static map after the static map is loaded into the memory. The map point pointer array is used for sequentially storing pointers of each map point structure in the static map after the static map is loaded into the memory. The descriptor array head address is used for storing head addresses of a plurality of continuous descriptor data structures after the static map is loaded into the memory. The descriptor management pointer queue is responsible for maintaining free descriptor locations in the map.

In some embodiments, the electronic device may create a key frame pointer array, a three-dimensional map point pointer array, and a description sub-array head address variable, and load map data of the visual feature point map to be compressed with the key frame pointer array, the three-dimensional map point pointer array, and the description sub-array head address variable as map handles.

It can be understood that by creating the key frame pointer array, the three-dimensional map point pointer array and the description sub-array head address variable, the pointer of each key frame data structure and the pointer of each three-dimensional map point data structure in the visual feature point map data to be compressed can be loaded into the memory, and the subsequent electronic equipment can directly operate different pointer data, so that the operation pressure of the electronic equipment is reduced.

S302, determining target feature points in a plurality of key frame data.

Wherein, the target feature points do not have corresponding three-dimensional map points in the map data.

As one possible implementation manner, the electronic device traverses each feature point in each key frame data, and determines whether each feature point has a corresponding three-dimensional map point. And if one feature point does not exist the corresponding three-dimensional map point, determining the feature point as a target feature point. And if one feature point exists a corresponding three-dimensional map point, determining the feature point as a non-target feature point.

In some embodiments, map point information corresponding to each feature point is also stored in each key frame data structure. For example, if a feature point exists in a map point, the key frame data structure may store a map point index corresponding to the feature point. If a feature point does not have a corresponding map point, the key frame data structure may store an invalid map point index (e.g., -1) corresponding to the feature point to indicate that the feature point does not have a corresponding map point.

In practical application, the electronic device may traverse each key frame data structure according to the pointer of each key frame data structure stored in the key frame pointer array, so as to read the map point index corresponding to the storage feature point in each key frame data structure, and further judge whether each feature point has a corresponding three-dimensional map point according to the map point index corresponding to the feature point. And if one feature point does not exist the corresponding three-dimensional map point, determining the feature point as a target feature point. And if one feature point exists a corresponding three-dimensional map point, determining the feature point as a non-target feature point.

S303, deleting the description sub-data of the target feature points from the plurality of description sub-data to obtain compressed map data.

As a possible implementation manner, the electronic device determines the descriptor data of the target feature point from the plurality of descriptor data, and deletes the descriptor data to obtain the compressed map data.

The map data of the to-be-compressed visual feature point map comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises the description sub-data of each feature point and the description sub-data of each three-dimensional map point, so that the key frame and description sub-sets contained in the map points can be conveniently managed. Further, the present application determines that feature points (i.e., target feature points) of corresponding three-dimensional map points do not exist in the plurality of key frame data, and deletes descriptor data of the feature points, so as to reduce the number of descriptor data. In this way, the redundant deletion of descriptors is realized by adopting the method for centrally managing the existing keyframes and the descriptors contained in the map points, so that the three-dimensional map points and the keyframes in the map are reserved while the data volume of the map data is reduced, and the influence on the map precision is low.

In one design, a plurality of reserved spaces are reserved in front of a plurality of continuously stored descriptor data structures, and one reserved space can accommodate one descriptor data. In order to keep compact arrangement of the compressed descriptor data, as shown in fig. 6, after deleting the descriptor data of the target feature point, the map data compression method provided in the embodiment of the present application may further include:

s401, creating an idle description sub-pointer queue, and adding pointers of a plurality of reserved spaces to the idle description sub-pointer queue.

The free descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors.

It should be noted that the reserved space is located in front of the descriptor array, and map compression is performed during map construction, so that the reserved space exists during map construction, and the electronic device can not reserve the space during subsequent map multiplexing. For example, the electronic device reserves a space of T descriptors, where T is the maximum of the number of descriptors stored in a single key frame.

S402, traversing each feature point in each key frame data, and determining a description sub pointer corresponding to the description sub data of each feature point.

In some embodiments, a key frame data further includes a descriptor sequence number (e.g., descriptor index) corresponding to each feature point in a key frame, where a feature point corresponds to a descriptor, and the descriptor sequence number is used to reflect a position of a descriptor corresponding to a feature point in a plurality of descriptor data. After loading map data of a visual feature point map to be compressed, the electronic equipment acquires a descriptor sequence number corresponding to a feature point for any one feature point. Further, the electronic device determines the position offset between the descriptor and the first address of the descriptor data structure in the descriptor array first address variable according to the size of one description data in the map data of the visual feature point to be compressed and the descriptor sequence number corresponding to the feature point. The electronic device determines a descriptor pointer corresponding to the descriptor data of the feature point based on the first address and the position offset of the descriptor data structure in the descriptor array first address variable.

Since the descriptors themselves, and not the corresponding pointers of the descriptor arrays, are stored in the key frames and the map points, but the subscripts, when a two-dimensional feature point in the key frames needs to index its corresponding descriptor vector, or the map points need to index its corresponding descriptor vector, the indexing logic thereof becomes as shown in fig. 7. Firstly, the electronic device obtains the size A of a single descriptor (namely, the size of each descriptor in a descriptor array), then obtains the index B of the corresponding descriptor (namely, the descriptor to be indexed) array stored in a key frame or map point and the head address P (contained in a dynamic map handle when loaded) of the descriptor array, and finally obtains the accessed descriptor pointer address by adopting the following calculation method:

Q＝P+A*B

it will be appreciated that the electronic device can obtain the desired descriptor vector via the pointer Q.

S403, judging whether each characteristic point has a corresponding three-dimensional map point.

The specific implementation of this step may refer to the content of S302, which is not described herein.

S404, if one feature point does not have a corresponding three-dimensional map point, adding a description sub pointer corresponding to description sub data of the feature point to an idle description sub pointer queue.

As a possible implementation manner, if a feature point does not have a corresponding three-dimensional map point, the electronic device considers that the descriptor belongs to a redundant descriptor, and directly places a pointer corresponding to the descriptor into an idle descriptor pointer queue.

S405, if a corresponding three-dimensional map point exists in a feature point, acquiring a pointer from an idle descriptor pointer queue, and after copying the descriptor data of the feature point to a memory address corresponding to the pointer, adding an original descriptor pointer corresponding to the descriptor data of the feature point to the idle descriptor pointer queue.

The memory address is any one reserved space or any memory address corresponding to a descriptor pointer added to the space pointer queue.

As a possible implementation manner, if a feature point has a corresponding three-dimensional map point, the electronic device obtains a pointer with the forefront storage position according to the storage position corresponding to each pointer in the idle description sub-pointer queue. Further, the electronic device copies the descriptor data of the feature points to the memory address corresponding to the pointer. After copying the descriptor data of the feature points to the memory address corresponding to the pointer, the electronic device adds an original descriptor pointer corresponding to the descriptor data of the feature points to the idle descriptor pointer queue.

As shown in FIG. 8, before map compression, the multiple sequentially stored descriptor data structures include both valid descriptors and redundant descriptors, and the descriptor arrays in the map occupy a larger space. After the map is compressed, the effective description subset is stored, so that the size of the map is reduced, and the efficiency of centralizing index descriptions in positioning is improved conveniently.

Similarly, the descriptor data for the three-dimensional map points may be similarly processed. Specifically, the electronic device may traverse each three-dimensional map point data and determine a descriptor pointer corresponding to descriptor data of each three-dimensional map point data. Further, for the description sub-data of any three-dimensional map point data, the electronic device acquires a pointer from the idle description sub-pointer queue, copies the description sub-data of the three-dimensional map point data to a memory address corresponding to the pointer, and adds an original description sub-pointer corresponding to the description sub-data of the three-dimensional map point data to the idle description sub-pointer queue so as to enable the subsequent description sub-data structure to use the original description sub-pointer.

In some embodiments, after copying the descriptor data of the feature point to the reserved space corresponding to the pointer, the electronic device may determine a current descriptor sequence number corresponding to the feature point based on the current pointer of the descriptor data of the feature point and a first address of the descriptor data structure in a first address variable of the descriptor array, and update an original descriptor sequence number corresponding to the feature point in the key frame data based on the current descriptor sequence number.

Similarly, the electronic device may determine a current descriptor sequence number corresponding to the three-dimensional map point based on the current pointer of the descriptor data of the three-dimensional map point data and the first address of the descriptor data structure in the first address variable of the descriptor array, and update an original descriptor sequence number corresponding to the three-dimensional map point in the three-dimensional map point data based on the current descriptor sequence number.

Therefore, when the electronic device uses the compressed map, the corresponding descriptor data can be determined in the compressed descriptor array by updating the descriptor array subscript corresponding to the feature point or the descriptor array subscript corresponding to the map point.

In another design, no reserved space is set before the plurality of continuously stored descriptor data structures, in this case, after deleting the descriptor data of the target feature point, the map data compression method provided in the embodiment of the present application may further include:

the electronic device creates an idle descriptor pointer queue. The free descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors. Further, the electronic device traverses each feature point in each key frame data, determines a descriptor pointer corresponding to the descriptor data of the current feature point for any traversed current feature point, and adds the descriptor pointer to the idle descriptor pointer queue. The electronic equipment judges whether the current feature point has a corresponding three-dimensional map point, if the current feature point has the corresponding three-dimensional map point, a pointer is obtained from an idle description sub pointer queue, the description sub data of the current feature point is copied to a memory address corresponding to the pointer, and the pointer is deleted from the idle description sub pointer queue. If the current feature point does not have the corresponding three-dimensional map point, the electronic equipment does not process, and continues to traverse the next feature point, and continues the process.

In some embodiments, after copying the descriptor data satisfying the condition to the memory address corresponding to the idle descriptor pointer queue pointer, the electronic device may further determine a location of a valid descriptor data structure in the plurality of continuously stored descriptor data structures; the descriptor data structure is not empty into a valid descriptor data structure. Further, the electronic device discards a storage space after the effective description sub-data structure in the plurality of continuously stored description sub-data structures to obtain a new plurality of continuously stored description sub-data structures; the number of new plurality of consecutively stored description sub-data structures is less than the number of plurality of consecutively stored description sub-data structures.

It can be understood that after deleting the description sub data of the target feature points, the electronic device can also retain the useful information at the front part in the description sub data structure and delete the redundant information at the rear part, thereby shortening the length of the description sub data structure and further reducing the size of the map data.

In some embodiments, based on the map architecture described above, the embodiments of the present application may also add new keyframes or map points to the map architecture.

Specifically, the electronic equipment responds to a key frame adding request and acquires an idle key frame pointer from an idle key frame pointer queue; the key frame adding request is used for requesting to insert a new key frame into the visual feature point map to be compressed, and the idle key frame pointer queue is used for collecting memory addresses which can be used for moving the key frames. Further, the electronic device copies the data except the description sub-data in the new key frame to the memory address corresponding to the idle key frame pointer, acquires an idle description sub-pointer from the idle description sub-pointer queue, copies the description sub-data in the new key frame to the memory address corresponding to the idle description sub-pointer, and stores the memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle key frame pointer.

It should be noted that, here, the key frame has several descriptors, and the electronic device needs to take several idle descriptor pointers. In practical application, the electronic device may first obtain a pointer, then copy a descriptor corresponding to a feature point, then obtain a pointer, and then copy a descriptor corresponding to another feature point. After copying is completed, the electronic device may also store the index (i.e., memory address information) corresponding to the descriptor in the key frame, so that the key frame may successfully index into the corresponding descriptor.

Similarly, the electronic equipment responds to the map point adding request to acquire an idle map point pointer from the idle map point pointer queue; the map point adding request is used for requesting to insert new map points into the visual feature point map to be compressed, and the idle map point pointer queue is used for collecting memory addresses which can be used for moving the map points. Further, the electronic device copies the data except the description sub-data in the new map point to the memory address corresponding to the idle map point pointer, acquires an idle description sub-pointer from the idle description sub-pointer queue, copies the description sub-data in the new map point to the memory address corresponding to the idle description sub-pointer, and stores the memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle map point pointer.

The foregoing description of the solution provided in the embodiments of the present application has been mainly presented in terms of a method. To achieve the above functions, it includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. The technical aim may be to use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the present application.

In an exemplary embodiment, the embodiment of the application also provides a map data compression device. Fig. 9 is a schematic diagram of the composition of the map data compression device according to the embodiment of the present application. As shown in fig. 9, the map data compression apparatus includes: an acquisition unit 501, a determination unit 502, and a processing unit 503.

An obtaining unit 501, configured to obtain map data of a to-be-compressed visual feature point map; the map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point; a determining unit 502, configured to determine target feature points in the plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data; the processing unit 503 is configured to delete the descriptor data of the target feature point from the plurality of descriptor data, and obtain compressed map data.

In one possible implementation manner, one key frame data corresponds to one key frame data structure, a plurality of key frame data corresponds to a plurality of continuously stored key frame data structures, one three-dimensional map point data corresponds to one three-dimensional map point data structure, a plurality of three-dimensional map point data corresponds to a plurality of continuously stored three-dimensional map point data structures, one descriptor data corresponds to one descriptor data structure, and a plurality of descriptor data corresponds to a plurality of continuously stored descriptor data structures; the acquiring unit 501 is specifically configured to: creating a key frame pointer array, a three-dimensional map point pointer array and a description sub-array head address variable; the key frame pointer array is used for storing pointers of each key frame data structure in the visual characteristic point map data to be compressed, the three-dimensional map point pointer array is used for storing pointers of each three-dimensional map point data structure in the visual characteristic point map data to be compressed, and the description sub-array head address variable is used for storing sequence head addresses of a plurality of continuous description sub-data structures; and taking the key frame pointer array, the three-dimensional map point pointer array and the description sub-array head address variable as map handles to load map data of the visual feature point map to be compressed.

In a possible implementation manner, after deleting the descriptor data of the target feature point, the processing unit 503 is further configured to: creating an idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data; for any traversed current feature point, determining a descriptor pointer corresponding to the descriptor data of the current feature point, and adding the descriptor pointer to an idle descriptor pointer queue; judging whether the current characteristic points have corresponding three-dimensional map points or not; if the corresponding three-dimensional map points exist in the current feature points, a pointer is obtained from the idle description sub-pointer queue, the description sub-data of the current feature points are copied to the memory addresses corresponding to the pointers, and the pointers are deleted from the idle description sub-pointer queue.

In one possible implementation, a plurality of reserved spaces are reserved in front of a plurality of continuously stored descriptor data structures, and one reserved space can accommodate one descriptor data; after deleting the descriptor data of the target feature point, the processing unit 503 is further configured to: creating an idle description sub-pointer queue, and adding pointers of a plurality of reserved spaces to the idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors; traversing each feature point in each key frame data, and determining a description sub pointer corresponding to the description sub data of each feature point; judging whether each characteristic point has a corresponding three-dimensional map point or not; if one feature point does not have a corresponding three-dimensional map point, adding a descriptor pointer corresponding to descriptor data of the feature point to an idle descriptor pointer queue; if one feature point has a corresponding three-dimensional map point, acquiring a pointer from an idle descriptor pointer queue, and adding an original descriptor pointer corresponding to descriptor data of the feature point to the idle descriptor pointer queue after copying the descriptor data of the feature point to a memory address corresponding to the pointer; the memory address is any one reserved space or any memory address corresponding to the descriptor pointer added to the idle pointer queue.

In a possible implementation manner, the processing unit 503 is specifically configured to: and acquiring the pointer which enters the idle description sub-pointer queue earliest according to the time when each pointer in the idle description sub-pointer queue enters the idle description sub-pointer queue.

In one possible implementation manner, the key frame data further includes a descriptor sequence number corresponding to each feature point in the key frame, one feature point corresponds to one descriptor, and the descriptor sequence number is used for reflecting the position of the descriptor corresponding to one feature point in the plurality of descriptor data; after loading the map data of the visual feature point map to be compressed, the processing unit 503 is specifically configured to: for any one feature point, a descriptor sequence number corresponding to the feature point is obtained; determining the position offset between the descriptor and the first address of the descriptor data structure in the descriptor array first address variable according to the size of one description data in the visual feature point map data to be compressed and the descriptor sequence number corresponding to the feature point; and determining a descriptor pointer corresponding to the descriptor data of the feature point based on the first address and the position bias of the descriptor data structure in the descriptor array first address variable.

In a possible implementation, the processing unit 503 is further configured to: traversing each three-dimensional map point data, and determining a description sub pointer corresponding to the description sub data of each three-dimensional map point data; for the description sub-data of any three-dimensional map point data, a pointer is obtained from an idle description sub-pointer queue, the description sub-data of the three-dimensional map point data is copied to a storage space corresponding to the pointer, and an original description sub-pointer corresponding to the description sub-data of the three-dimensional map point data is added to the idle description sub-pointer queue so that the original description sub-pointer can be used by a subsequent description sub-data structure.

In a possible implementation manner, after copying the descriptor data of the feature point into the reserved space corresponding to the pointer, the processing unit 503 is further configured to: based on the current pointer of the descriptor data of the feature points and the first address of the descriptor data structure in the descriptor array first address variable, the current descriptor sequence number corresponding to the feature points is determined, and the original descriptor sequence number corresponding to the feature points in the key frame data is updated based on the current descriptor sequence number.

In a possible implementation manner, one three-dimensional map point corresponds to one descriptor data, one three-dimensional map point data further comprises a descriptor sequence number corresponding to the three-dimensional map point, and the descriptor sequence number is used for reflecting the position of the descriptor data corresponding to the three-dimensional map point in the plurality of descriptor data; after copying the descriptor data of the three-dimensional map point data to the reserved space corresponding to the pointer, the processing unit 503 is further configured to: based on the current pointer of the description sub data of the three-dimensional map point data and the first address of the description sub data structure in the description sub array first address variable, determining the current description sub sequence number corresponding to the three-dimensional map point, and based on the current description sub sequence number, updating the original description sub sequence number corresponding to the three-dimensional map point in the three-dimensional map point data.

In a possible implementation manner, after copying the descriptor data satisfying the condition to the memory address corresponding to the free descriptor pointer queue pointer, the processing unit 503 is further configured to: determining the position of a valid description sub-data structure in a plurality of continuously stored description sub-data structures; the descriptor data structure is not empty to be a valid descriptor data structure; discarding storage space after the effective description sub-data structure in the plurality of continuously stored description sub-data structures to obtain a new plurality of continuously stored description sub-data structures; the number of new plurality of consecutively stored description sub-data structures is less than the number of plurality of consecutively stored description sub-data structures.

In a possible implementation, the processing unit 503 is further configured to: responding to a key frame adding request, and acquiring an idle key frame pointer from an idle key frame pointer queue; the key frame adding request is used for requesting to insert a new key frame into the visual feature point map to be compressed, and the idle key frame pointer queue is used for collecting memory addresses which can be used for moving the key frames; copying data except the description sub data in the new key frame to a memory address corresponding to the idle key frame pointer; and acquiring an idle description sub-pointer from the idle description sub-pointer queue, copying description sub-data in the new key frame to a memory address corresponding to the idle description sub-pointer, and storing the memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle key frame pointer.

In a possible implementation, the processing unit 503 is further configured to: responding to a map point adding request, and acquiring an idle map point pointer from an idle map point pointer queue; the map point adding request is used for requesting to insert new map points into the visual feature point map to be compressed, and the idle map point pointer queue is used for collecting memory addresses which can be used for moving the map points; copying data except the description sub data in the new map point to a memory address corresponding to the idle map point pointer; and acquiring an idle description sub pointer from the idle description sub pointer queue, copying description sub data in the new map point to a memory address corresponding to the idle description sub pointer, and storing the memory address information corresponding to the idle description sub pointer to the memory address corresponding to the idle map point pointer.

It should be noted that the division of the modules in fig. 9 is illustrative, and is merely a logic function division, and other division manners may be actually implemented. For example, two or more functions may also be integrated in one processing module. The integrated modules may be implemented in hardware or in software functional units.

In an exemplary embodiment, a computer readable storage medium is also provided, comprising software instructions which, when run on an electronic device, cause the electronic device to perform any of the methods provided by the above embodiments.

In an exemplary embodiment, the present application also provides a computer program product comprising computer-executable instructions, which, when run on an electronic device, cause the electronic device to perform any of the methods provided by the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer-executable instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, from one website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be accessed by a computer or data storage devices including one or more servers, data centers, etc. that can be integrated with the media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a Solid State Disk (SSD), etc.

Although the present application has been described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the figures, the disclosure, and the appended claims. In the claims, the word "Comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Although the present application has been described in connection with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely exemplary illustrations of the present application as defined in the appended claims and are considered to cover any and all modifications, variations, combinations, or equivalents that fall within the scope of the present application. It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (16)

1. A method of map data compression, the method comprising:

acquiring map data of a to-be-compressed visual feature point map; the map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point;

determining target feature points in the plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data;

and deleting the description sub-data of the target feature points from the plurality of description sub-data to obtain compressed map data.

2. The method of claim 1, wherein one key frame data corresponds to one key frame data structure, the plurality of key frame data corresponds to a plurality of consecutively stored key frame data structures, one three-dimensional map point data corresponds to one three-dimensional map point data structure, the plurality of three-dimensional map point data corresponds to a plurality of consecutively stored three-dimensional map point data structures, one descriptor data corresponds to one descriptor data structure, and the plurality of descriptor data corresponds to a plurality of consecutively stored descriptor data structures; the obtaining the map data of the to-be-compressed visual feature point map comprises the following steps:

creating a key frame pointer array, a three-dimensional map point pointer array and a description sub-array head address variable; the key frame pointer array is used for storing pointers of each key frame data structure in the to-be-compressed visual feature point map data, the three-dimensional map point pointer array is used for storing pointers of each three-dimensional map point data structure in the to-be-compressed visual feature point map data, and the descriptor array head address variable is used for storing sequence head addresses of the plurality of continuous descriptor data structures;

and taking the key frame pointer array, the three-dimensional map point pointer array and the first address variable of the description sub-array as map handles to load map data of the to-be-compressed visual feature point map.

3. The method of claim 2, wherein after deleting the descriptor data for the target feature point, the method further comprises:

creating an idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors;

traversing each feature point in each key frame data;

for any traversed current feature point, determining a descriptor pointer corresponding to descriptor data of the current feature point, and adding the descriptor pointer to the idle descriptor pointer queue;

judging whether the current feature points have corresponding three-dimensional map points or not;

if the current feature point has a corresponding three-dimensional map point, a pointer is obtained from the idle description sub-pointer queue, the description sub-data of the current feature point is copied to a memory address corresponding to the pointer, and the pointer is deleted from the idle description sub-pointer queue.

4. The method of claim 2, wherein a plurality of reserved spaces are reserved before a plurality of serially stored descriptor data structures, one reserved space accommodating one descriptor data; after deleting the descriptor data of the target feature point, the method further includes:

Creating an idle description sub-pointer queue, and adding pointers of the plurality of reserved spaces to the idle description sub-pointer queue; the idle descriptor pointer queue is used for collecting memory addresses which can be used for moving descriptors;

traversing each feature point in each key frame data, and determining a description sub pointer corresponding to the description sub data of each feature point;

judging whether each characteristic point has a corresponding three-dimensional map point or not;

if one feature point does not have a corresponding three-dimensional map point, adding a description sub-pointer corresponding to description sub-data of the feature point to the idle description sub-pointer queue;

if one feature point has a corresponding three-dimensional map point, acquiring a pointer from the idle descriptor pointer queue, and adding an original descriptor pointer corresponding to the descriptor data of the feature point to the idle descriptor pointer queue after copying the descriptor data of the feature point to a memory address corresponding to the pointer; the memory address is any one reserved space or any memory address corresponding to the descriptor pointer added to the idle pointer queue.

5. The method of claim 4, wherein said obtaining a pointer from said free descriptor pointer queue comprises:

And acquiring the pointer which enters the idle description sub-pointer queue earliest according to the time when each pointer in the idle description sub-pointer queue enters the idle description sub-pointer queue.

6. The method of claim 4, wherein a key frame data further comprises a descriptor sequence number corresponding to each feature point in a key frame, one feature point corresponding to one descriptor, and the descriptor sequence number is used to reflect a position of a descriptor corresponding to one feature point in the plurality of descriptor data; after loading the map data of the to-be-compressed visual feature point map, determining a description sub pointer corresponding to the description sub data of each feature point, including:

for any one feature point, acquiring a descriptor sequence number corresponding to the feature point;

determining the position offset between the descriptor and the first address of the descriptor data structure in the descriptor array first address variable according to the size of one description data in the visual feature point map data to be compressed and the descriptor sequence number corresponding to the feature point;

and determining a descriptor pointer corresponding to the descriptor data of the feature point based on the first address of the descriptor data structure in the descriptor array first address variable and the position offset.

7. The method according to claim 4, wherein the method further comprises:

traversing each three-dimensional map point data, and determining a description sub pointer corresponding to the description sub data of each three-dimensional map point data;

for the description sub-data of any three-dimensional map point data, a pointer is obtained from the idle description sub-pointer queue, the description sub-data of the three-dimensional map point data is copied to a storage space corresponding to the pointer, and an original description sub-pointer corresponding to the description sub-data of the three-dimensional map point data is added to the idle description sub-pointer queue for the subsequent description sub-data structure to use the original description sub-pointer.

8. The method of claim 6, wherein after copying the descriptor data of the feature point into the reserved space corresponding to the pointer, the method further comprises:

and determining a current descriptor sequence number corresponding to the feature point based on a current pointer of the descriptor data of the feature point and a first address of a descriptor data structure in a first address variable of the descriptor array, and updating an original descriptor sequence number corresponding to the feature point in key frame data based on the current descriptor sequence number.

9. The method of claim 7, wherein one three-dimensional map point corresponds to one descriptor data, and one three-dimensional map point data further comprises a descriptor sequence number corresponding to one three-dimensional map point, the descriptor sequence number being used to reflect a position of the descriptor data corresponding to one three-dimensional map point in the plurality of descriptor data; after copying the description sub-data of the three-dimensional map point data to the reserved space corresponding to the pointer, the method further comprises:

determining a current descriptor sequence number corresponding to the three-dimensional map point based on a current pointer of the descriptor data of the three-dimensional map point data and a first address of a descriptor data structure in a first address variable of the descriptor array, and updating an original descriptor sequence number corresponding to the three-dimensional map point in the three-dimensional map point data based on the current descriptor sequence number.

10. The method according to claim 3 or 4, wherein after copying the descriptor data satisfying the condition to the memory address corresponding to the free descriptor pointer queue pointer, the method further comprises:

determining the position of a valid descriptor data structure in the plurality of continuously stored descriptor data structures; the descriptor data structure is not empty to be a valid descriptor data structure;

Discarding storage space after the effective description sub-data structure in the plurality of continuously stored description sub-data structures to obtain a new plurality of continuously stored description sub-data structures; the number of the new plurality of consecutively stored descriptor data structures is less than the number of the plurality of consecutively stored descriptor data structures.

11. The method according to claim 3 or 4, characterized in that the method further comprises:

responding to a key frame adding request, and acquiring an idle key frame pointer from an idle key frame pointer queue; the key frame adding request is used for requesting to insert a new key frame into the visual feature point map to be compressed, and the idle key frame pointer queue is used for collecting memory addresses which can be used for moving the key frames;

copying data except the description sub-data in the new key frame to a memory address corresponding to the idle key frame pointer;

and acquiring an idle description sub-pointer from the idle description sub-pointer queue, copying description sub-data in the new key frame to a memory address corresponding to the idle description sub-pointer, and storing memory address information corresponding to the idle description sub-pointer to the memory address corresponding to the idle key frame pointer.

12. The method according to claim 3 or 4, characterized in that the method further comprises:

responding to a map point adding request, and acquiring an idle map point pointer from an idle map point pointer queue; the map point adding request is used for requesting to insert new map points into the visual feature point map to be compressed, and the idle map point pointer queue is used for collecting memory addresses which can be used for moving the map points;

copying data except the description sub data in the new map point to a memory address corresponding to the idle map point pointer;

and acquiring an idle description sub pointer from the idle description sub pointer queue, copying description sub data in the new map point to a memory address corresponding to the idle description sub pointer, and storing memory address information corresponding to the idle description sub pointer to the memory address corresponding to the idle map point pointer.

13. The method of claim 2, wherein the determining the target feature point in the plurality of keyframe data comprises:

traversing each key frame data structure according to the pointer of each key frame data structure stored in the key frame pointer array so as to read map point subscripts corresponding to the feature points stored in each key frame data structure;

Judging whether each feature point has a corresponding three-dimensional map point according to the map point subscript corresponding to the feature point, and if one feature point does not have a corresponding three-dimensional map point, determining the feature point as a target feature point.

14. A map data compression apparatus, characterized in that the apparatus comprises an acquisition unit, a determination unit, and a processing unit;

the acquisition unit is used for acquiring map data of the visual feature point map to be compressed; the map data of the visual feature point map to be compressed comprises a plurality of key frame data, a plurality of three-dimensional map point data and a plurality of description sub-data, wherein one key frame data comprises coordinates of a plurality of feature points in one key frame, one three-dimensional map point data comprises coordinates of one three-dimensional map point and associated information of corresponding feature points of the three-dimensional map point in the key frame, and the plurality of description sub-data comprises description sub-data of each feature point and description sub-data of each three-dimensional map point;

the determining unit is used for determining target feature points in the plurality of key frame data; the target feature points do not have corresponding three-dimensional map points in the map data;

And the processing unit is used for deleting the description sub-data of the target characteristic points from the plurality of description sub-data to obtain compressed map data.

15. An electronic device, comprising: a processor and a memory;

the memory stores instructions executable by the processor;

the processor is configured to, when executing the instructions, cause the electronic device to implement the method of any one of claims 1-13.

16. A computer-readable storage medium, the readable storage medium comprising: a software instruction;

the software instructions, when executed in an electronic device, cause the electronic device to implement the method of any one of claims 1-13.