Disclosure of Invention
The embodiment of the application provides a three-dimensional map display method and a mobile terminal, and realizes a method for receiving, analyzing and displaying a three-dimensional probability map in real time.
In a first aspect, an embodiment of the present application provides a three-dimensional map display method, including:
receiving a binary data stream of the three-dimensional point cloud probability map data sent by the server, dividing each 8 bytes of the binary data stream into a data segment to obtain N data segments, wherein N is an integer greater than 1;
according to a recursive algorithm, sequentially traversing N data segments by using a data structure of an octree, and analyzing node information of each data segment according to eight data unit information in each data segment;
acquiring octree node information corresponding to the display level according to the display level of the three-dimensional point cloud probability map;
and displaying the three-dimensional map on a user interface according to the acquired node information.
Optionally, analyzing the node information of each data segment according to the eight data unit information in each data segment, specifically including:
acquiring eight data unit information of each layer of first node in the octree, and acquiring color information of the first node and child node information of the first node according to effective bits in the eight data unit information;
if the child node information of the first node indicates that the first node has child nodes, acquiring the information of the next-layer child nodes of the first node; and if the child node information of the first node indicates that the first node does not have child nodes, acquiring second node information of the layer number where the first node is located.
Optionally, the node information includes:
the number of layers of the nodes, the three-dimensional coordinates of the nodes, the color information of the nodes and the information of the nodes at the lower layer of the nodes.
Optionally, before acquiring octree node information corresponding to a display level according to the display level of the three-dimensional point cloud probability map, the method further includes:
and determining the display level of the three-dimensional point cloud probability map according to the input of a user.
In a second aspect, an embodiment of the present application provides a mobile terminal, including:
the receiving module is used for receiving a binary data stream of the three-dimensional point cloud probability map data sent by the server;
the processing module is used for dividing each 8 bytes of the binary data stream into a data segment to obtain N data segments, wherein N is an integer greater than 1; sequentially traversing the N data segments by using the data structure of the octree according to a recursive algorithm, and analyzing the node information of each data segment according to the eight data unit information in each data segment; acquiring octree node information corresponding to the display level according to the display level of the three-dimensional point cloud probability map;
and the display module is used for displaying the three-dimensional map on the user interface according to the acquired node information.
Optionally, the processing module is specifically configured to:
acquiring eight data unit information of each layer of first node in the octree, and acquiring color information of the first node and child node information of the first node according to effective bits in the eight data unit information;
if the child node information of the first node indicates that the first node has child nodes, acquiring the information of the next-layer child nodes of the first node; and if the child node information of the first node indicates that the first node does not have child nodes, acquiring second node information of the layer number where the first node is located.
Optionally, the node information includes:
the number of layers of the nodes, the three-dimensional coordinates of the nodes, the color information of the nodes and the information of the nodes at the lower layer of the nodes.
Optionally, the processing module is further configured to:
and determining the display level of the three-dimensional point cloud probability map according to the input of a user before acquiring octree node information corresponding to the display level according to the display level of the three-dimensional point cloud probability map.
In a third aspect, an embodiment of the present application provides a mobile terminal, including: a processor and a memory;
a memory coupled to the processor and configured to store computer instructions; a processor, coupled to the memory, configured to execute the computer instructions to cause the mobile terminal to perform the method of any of the first aspects.
In a fourth aspect, embodiments of the present application provide a computer storage medium having computer program instructions stored therein, which when executed on a computer, cause the computer to perform the method of any of the above first aspects.
In the embodiments of the present application, the three-dimensional map display method applied to the mobile terminal may receive and analyze the three-dimensional probability map based on the octree format in real time through the network and perform synchronous display on the mobile terminal.
Detailed Description
The embodiment of the application provides a three-dimensional map display method and a mobile terminal, wherein the mobile terminal can receive and analyze a three-dimensional probability map based on an octree format in real time through a network and synchronously display the three-dimensional map at the mobile terminal.
To make the objects, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions of the exemplary embodiments of the present application will be clearly and completely described below with reference to the drawings in the exemplary embodiments of the present application.
Fig. 1 schematically shows an application scene architecture diagram of a three-dimensional map display provided in an embodiment of the present application. As shown in fig. 1, includes: terminal 100, network 200, server 300.
The terminal 100 can be a smart phone, a tablet computer, a vehicle-mounted terminal, a wearable device and the like of an android platform, can display a three-dimensional map in a display screen in real time, and can allow a user to select a display effect of the three-dimensional map through touch screen operation.
The network 200 may be the internet, a local area network, the internet, etc. for connecting data communication between the terminal 100 and the server 300.
The server 300 may be a common web server, enterprise server, or the like, for transmitting a binary data stream of three-dimensional point cloud probability map data in real time.
For clarity of description of embodiments of the present application, octrees will be described in detail below. Fig. 2 illustrates an octree structure diagram provided in an embodiment of the present application.
As shown in fig. 2, the three-dimensional space model is divided into a number of small blocks, and if each face of each small block is cut into two pieces, the small blocks become eight small blocks of the same size. This step is repeated until the final square size reaches the highest accuracy of modeling. In the process, the "expansion from one node to eight child nodes" is performed, and then the process of subdividing the whole space from the maximum space to the minimum space is an octree. The entire large square can be considered the root node, while the smallest block can be considered the "leaf node". Thus, in an octree, when one level of nodes goes down from the upper level, the volume of the map is reduced to eight times of the original volume.
For example, if the leaf node square size is 1 square centimeter, then when the octree is limited to 10 levels, the total modelable volume is approximately 10 cubic centimeters to the power of 8, i.e., 1073 square meters. Since volume is exponential with depth, the modeled volume grows very fast when larger depths are used.
In the octree, information on whether or not a node is occupied is stored in the node. When all children nodes of a block are occupied or not, it is not necessary to expand the node. For example, only one root node is needed at the beginning when the map is empty, and the complete tree is not needed. When adding information to a map, most octree nodes do not need to be spread to the leaf level, since actual objects are often connected together and blank places are also connected together.
The octree map with different resolutions stored in the form of octree consists of a plurality of small blocks, and when the resolution is higher, the blocks are very small; at lower resolutions, the square is large. The larger the resolution, the larger the number of layers to be analyzed.
In an octree, nodes store color information in RGB format.
Fig. 3 is a flowchart illustrating a three-dimensional map display method provided by an embodiment of the present application, where the flowchart includes the following steps:
s301: and the mobile terminal receives a binary data stream of the three-dimensional point cloud probability map data sent by the server.
In this step, the server may send a three-dimensional point cloud probability map created by an OctoMap-based function package in a Robot Operating System (ROS) to the mobile terminal in the form of Topic. The mobile terminal subscribes the three-dimensional point cloud probability map data through the application program to obtain a binary data stream of the three-dimensional point cloud probability map data.
S302: the mobile terminal divides each 8 bytes of the received binary data stream into a data segment to obtain N data segments, wherein N is an integer greater than 1.
S303: and the mobile terminal sequentially traverses the N data sections by using the data structure of the octree according to a recursive algorithm, and analyzes the node information of each data section according to the eight data unit information in each data section.
In this step, the node information of each data segment is analyzed according to the eight data unit information in each data segment, and the method specifically includes:
acquiring eight data unit information of each layer of first node in the octree, and acquiring color information of the first node and child node information of the first node according to effective bits in the eight data unit information;
if the child node information of the first node indicates that the first node has child nodes, acquiring the information of the next-layer child nodes of the first node; and if the child node information of the first node indicates that the first node does not have child nodes, acquiring second node information of the layer number where the first node is located.
S304: and the mobile terminal acquires octree node information corresponding to the display level according to the display level of the three-dimensional point cloud probability map.
In this step, the node information of the octree includes the number of layers of nodes, the three-dimensional coordinates of the nodes, the color information of the nodes, and the information of the nodes on the lower layer of the nodes. Through the data structure analysis of the octree, if the node has a child node, the size of the child node is 1/2 of the size of the node at the previous level, and the coordinates, the size and the number of layers of the child node at the next level can be determined according to the coordinates, the size and the number of layers of the node at the previous level.
For example, if the size of the root node is 1, the number of layers of the root node is 1, and the cube center of the root node is the origin of three-dimensional coordinates (0,0,0), the size of the eight character nodes of the root node is 1/2, the number of layers of the eight byte points of the root node is 2, and the three-dimensional coordinates of the eight character nodes of the root node are (0.25,0.25,0.25), (0,25, -0.25,0.25), (0.25, -0.25, -0.25, -0.25), (0.25,0.25,0.25), and so on, the coordinates of the eight character nodes of each child node can also be calculated.
Specifically, assuming that the central coordinate of the level node is (x, y, z,), the side length is length, and the number of layers is rank, the side length of eight byte points of the level node is length/2, the number of layers is rank +1, and the coordinates are:
(x+length/4,y+length/4,z+length/4);
(x-length/4,y+length/4,z+length/4);
(x+length/4,y-length/4,z+length/4);
(x-length/4,y-length/4,z+length/4);
(x+length/4,y+length/4,z-length/4);
(x-length/4,y+length/4,z-length/4);
(x+length/4,y-length/4,z-length/4);
(x-length/4,y-length/4,z-length/4)。
s305: and the mobile terminal displays the three-dimensional map on the user interface according to the acquired node information.
According to the above process, in some embodiments, a user may input the number of layers displayed by a map through a display screen of the mobile terminal; accordingly, in S304, the mobile terminal may obtain node information of the level according to the number of layers input by the user, including information such as size, color, and coordinates of the node; and drawing a three-dimensional point cloud probability map according to the node information of the level and displaying the three-dimensional point cloud probability map on a screen. The larger the number of layers input by the user is, the more the number of acquired nodes is, and the clearer the map is displayed, namely the display effect of the three-dimensional point cloud probability map can be adjusted by the user through the input number of layers.
For example, if the maximum depth of the octree is 16 (i.e. the octree includes 16 levels from top to bottom), and the display level includes 16 levels, the relationship between the display level and the octree depth can be as shown in table 1:
TABLE 1 relationship between map display level and OctreeDepthImage
Displaying a level
|
Number of octree levels
| Corresponding node |
|
1
|
1
|
Octree layer 1 node (root node)
|
2
|
2
|
Octree layer 2 node
|
3
|
3
|
Octree layer 3 node
|
……
|
……
|
…… |
According to table 1, one display level may correspond to one hierarchy number, and the higher the display level is, the greater the corresponding hierarchy number is. Taking display level 3 as an example, it corresponds to three levels of nodes, i.e., the third level of nodes of the octree.
In the embodiment of the application, the mobile terminal receives, analyzes and displays the three-dimensional point cloud probability map in real time through the network, convenience for a user to check the three-dimensional map effect in real time is improved, meanwhile, the user can adjust the display effect of the three-dimensional map through the display screen, and operability of the user is improved.
Fig. 4 illustrates a binary bit flow diagram of each node in an octree according to an embodiment of the present application. As shown in fig. 4, every 8 bytes in the binary data stream represents a node, the 25 th bit and the last 24 bits and the last 8 bits are valid data bits, the 25 th bit represents that the node is not occupied when the 25 th bit is 1, the node is occupied when the 25 th bit is 0, the node can be continuously divided, the last 24 bits represent RGB color information of the node, and each bit in the last 8 bits represents whether there is a child node.
For clearly explaining the three-dimensional map display method according to the embodiment of the present application, fig. 5 exemplarily shows a schematic diagram of a three-dimensional map parsing method according to the embodiment of the present application. The mobile terminal analyzes the received binary data stream of the three-dimensional point cloud data in an octree data structure, and can acquire the cubic space of each level of sub-nodes by adopting a recursive algorithm.
Parsing a binary data stream in an octree data format, comprising:
step 1: setting the maximum layer number of the subnodes of the octree, namely the maximum depth of the octree;
step 2: obtaining the maximum size of the three-dimensional map according to the received binary data stream, and determining the size of the cubic space of the octree root node;
and step 3: sequentially reading data segments in the data stream, and sequentially storing the data segments in a cube which takes a root node as an initial node and has no child nodes in the octree;
and 4, step 4: and determining whether the number of layers of the node reaches the maximum depth of the octree, stopping the division of the node if the number of layers of the node reaches the maximum depth of the octree, and dividing the node into sub-nodes which equally divide the cubic space of the node if the number of layers of the node does not reach the maximum depth of the octree.
In step 4, it may be determined whether the child node is occupied according to the 25 th bit number of the child node, and the node is subdivided into child nodes, and the number of child nodes in the next layer of the child node is determined according to the last 8 bit number of the child node.
Taking traversing the second-layer node of the octree as an example, firstly reading the first node of the second layer, judging whether the 25 th bit of the node is 0, if so, indicating that the node has a child node, then obtaining the color information of the node according to the last 24 bits of the node, finally obtaining the number of child nodes of the node according to the last 8 bits, if so, sequentially traversing the next-layer child nodes of the node, and increasing the depth value by 1, namely, increasing the number of layers of the node by 1, and so on until the depth value is equal to the set maximum depth value, reading the second node of the second layer. And circulating the above processes to traverse each data segment of the binary data stream.
Based on the same technical concept, the embodiment of the application also provides the mobile terminal. The mobile terminal performs the method in the above embodiment.
Fig. 6 illustrates an exemplary structure of a mobile terminal system according to an embodiment of the present application. As shown in fig. 6, includes: a receiving module 101, a processing module 102 and a display module 103.
The receiving module 101 is configured to receive a binary data stream of three-dimensional point cloud probability map data sent by a server;
the processing module 102 is configured to divide each 8 bytes of the binary data stream into one data segment to obtain N data segments, where N is an integer greater than 1; sequentially traversing the N data segments by using the data structure of the octree according to a recursive algorithm, and analyzing the node information of each data segment according to the eight data unit information in each data segment; acquiring octree node information corresponding to the display level according to the display level of the three-dimensional point cloud probability map;
and the display module 103 is configured to display the three-dimensional map on the user interface according to the acquired node information.
Optionally, the processing module is specifically configured to:
acquiring eight data unit information of each layer of first node in the octree, and acquiring color information of the first node and child node information of the first node according to effective bits in the eight data unit information;
if the child node information of the first node indicates that the first node has child nodes, acquiring the information of the next-layer child nodes of the first node; and if the child node information of the first node indicates that the first node does not have child nodes, acquiring second node information of the layer number where the first node is located.
Optionally, the node information includes the number of layers of the node, the three-dimensional coordinate of the node, the color information of the node, and the information of the node on the lower layer of the node.
Fig. 7 illustrates a three-dimensional map display effect diagram of the mobile terminal 100 according to an embodiment of the present application. As shown in fig. 7, a display effect 701 of the three-dimensional map is displayed on the display screen of the mobile terminal 100, and the zoom-in function key 703, the zoom-out function key 704, the cancel function key 705, and the refresh function key 706 may be folded by the function keys 702, and the display effect of the three-dimensional map may be adjusted by a sliding bar 707 representing the number of node layers, where the number of the sliding bar represents the number of node layers of the octree.
For example, when the user touches the function key 702 for the first time, the hidden zoom-in function key 703, zoom-out function key 704, cancel function key 705, and refresh function key 706 are displayed in the menu bar on the right side of the display screen, and when the user touches the function key 702 for the second time, the zoom-in function key 703, zoom-out function key 704, cancel function key 705, and refresh function key 706 are displayed in the menu bar on the right side of the display screen. Wherein the user can cancel the latest one-time operation through the cancel function key 705 and return to the initial effect map of the first three-dimensional map display through the refresh function key 706. The user can also input the number of layers of the octree nodes through the sliding bar 707 to adjust the display effect of the map, and the larger the number of input layers is, the more the number of acquired nodes is, and the clearer the display effect of the three-dimensional map is.
Optionally, the user may also directly touch the display screen to zoom in or out of the map or select a map area.
Embodiments of the present application also provide a computer storage medium for storing instructions that, when executed, may perform the method of the foregoing embodiments.
The embodiments of the present application also provide a computer program product for storing a computer program, where the computer program is used to execute the method of the foregoing embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.