CN115096286A

CN115096286A - Map merging method, apparatus, robot, storage medium, and program product

Info

Publication number: CN115096286A
Application number: CN202210689717.6A
Authority: CN
Inventors: 赵静; 何科君
Original assignee: Shenzhen Pudu Technology Co Ltd
Current assignee: Shenzhen Pudu Technology Co Ltd
Priority date: 2022-06-17
Filing date: 2022-06-17
Publication date: 2022-09-23

Abstract

The application relates to a map merging method, a map merging device, a robot, a storage medium and a program product. The method comprises the following steps: acquiring a plurality of local point cloud maps in a target area, wherein a superposition area exists between the local point cloud maps, and each local point cloud map comprises node information for constructing the local point cloud maps; determining a relative pose transformation relation between initial nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map; and performing coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation relation between the initial nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area. By adopting the method, the complexity of the method for combining a plurality of local point cloud maps into the global point cloud map can be reduced.

Description

Map merging method, apparatus, robot, storage medium, and program product

Technical Field

The present application relates to the field of robot positioning and navigation technologies, and in particular, to a map merging method, apparatus, robot, storage medium, and program product.

Background

Generally, data of a target area may be collected by a robot moving in the target area, and then the collected data is analyzed to construct a point cloud map of the target area.

In the related art, for a target area with a large range, the target area is divided into a plurality of sub-areas, the robot acquires a point cloud map corresponding to each sub-area by acquiring data information of each sub-area, a pose map is constructed by the point cloud map corresponding to each sub-area, and the local point cloud maps of each target sub-area are combined into a Global point cloud map according to a Global Positioning System (GPS) installed in the robot.

However, the method for merging a plurality of local point cloud maps into a global point cloud map in the related art is high in complexity.

Disclosure of Invention

In view of the above, it is desirable to provide a map merging method, an apparatus, a robot, a storage medium, and a program product capable of reducing the complexity of a method of merging a plurality of local point cloud maps into a global point cloud map, in view of the above technical problems.

In a first aspect, the present application provides a map merging method applied to a robot, the method including:

acquiring a plurality of local point cloud maps in a target area, wherein a superposition area exists between the local point cloud maps, and each local point cloud map comprises node information for constructing the local point cloud maps;

determining a relative pose transformation relation between initial nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map;

and performing coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation relation between the initial nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area.

In one embodiment, determining a relative pose transformation relationship between starting nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map comprises:

determining all adjacent local point cloud maps in each local point cloud map according to the overlapping area of each local point cloud map;

acquiring matching node pairs in adjacent local point cloud maps according to node information in each local point cloud map;

acquiring a node loop constraint relation between each matching node pair in each adjacent local point cloud map;

and determining a relative pose transformation relation between the initial nodes in each adjacent local point cloud map according to the node loop constraint relation between each matching node pair in each adjacent local point cloud map.

In one embodiment, if each adjacent local point cloud map comprises a first local point cloud map and a second local point cloud map; and the node information in each local point cloud map comprises point cloud descriptors of nodes;

then, according to the node information in each local point cloud map, obtaining matching node pairs in adjacent local point cloud maps, including:

for each adjacent local point cloud map, respectively matching the point cloud descriptors of the reference nodes in the second local point cloud map with the point cloud descriptors of the loop candidate nodes in the first local point cloud map,

and determining the reference node and the loopback candidate node which are successfully matched with the point cloud descriptor as a matching node pair to obtain the matching node pair in each adjacent local point cloud map.

In one embodiment, obtaining a node loopback constraint relationship between each matching node pair in each adjacent local point cloud map comprises:

aiming at any matching node pair, acquiring a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matched with each loop candidate node;

carrying out point cloud registration on the matched loop candidate local point cloud map and the reference local point cloud map;

and after point cloud registration is completed, determining the relative pose between the loop candidate node and the reference node in each matching node pair as the node loop constraint relation between each matching node pair.

In one embodiment, obtaining a loopback candidate local point cloud map corresponding to each loopback candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matched with each loopback candidate node comprises:

acquiring a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node;

splicing the loop candidate frames and a preset number of key frames according to poses of the key frames of the preset number adjacent to the front and back of each loop candidate frame to obtain a loop candidate local point cloud map corresponding to each loop candidate node;

and converting each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference nodes matched with each loop candidate node.

In one embodiment, determining the relative pose transformation between the start nodes in each adjacent local point cloud map according to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map comprises:

obtaining root mean square error values between all matching node pairs in each adjacent local point cloud map;

obtaining a target matching node pair with the minimum root mean square error value;

determining a node loop constraint relation between a reference node and a loop candidate node in a target matching node pair as relative pose transformation between initial nodes in each adjacent local point cloud map; and the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

In one embodiment, the method further comprises:

obtaining a difference value between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

and eliminating the matching node pairs with the difference values larger than the preset difference value to obtain candidate matching node pairs.

In one embodiment, the coordinate alignment conversion is performed on all node information in each local point cloud map based on the relative pose transformation between the starting nodes in each adjacent local point cloud map to obtain a global point cloud map of a target area, and the method includes:

converting all node information in each local point cloud map into the same coordinate system according to relative pose transformation between initial nodes in each adjacent local point cloud map to obtain an initial global point cloud map of a target area;

and optimizing the pose of the nodes in the initial global point cloud map through a preset optimizer to obtain the global point cloud map of the target area.

In one embodiment, optimizing the pose of the node in the initial global point cloud map by using a preset optimizer to obtain a global point cloud map of a target area, including:

inputting the node pose and the loop constraint relation in the candidate point cloud map into a preset optimizer, and optimizing the node pose through the optimizer to obtain a target node pose; the loop constraint comprises a node loop constraint relation between candidate matching node pairs, a loop constraint relation in each local point cloud map and a node loop constraint relation of adjacent nodes in each local point cloud map;

and constructing a global point cloud map of the target area based on the pose of the target node and the point cloud data corresponding to each node.

In one embodiment, obtaining a plurality of local point cloud maps in a target area comprises:

constructing a pose graph corresponding to each initial local point cloud map according to the initial local point cloud maps of a plurality of areas in the target area;

and transforming the point cloud data of each node to the global coordinate system of each initial local point cloud map through the node pose of each node in the pose map to obtain each local point cloud map.

In a second aspect, the present application further provides a map merging apparatus, including:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a plurality of local point cloud maps in a target area, a superposition area exists between the local point cloud maps, and each local point cloud map comprises node information of a robot in the process of constructing the local point cloud map;

the determining module is used for determining relative pose transformation between initial nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map;

and the conversion module is used for carrying out coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the initial nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area.

In a third aspect, the present application further provides a robot, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the method steps in any of the embodiments of the first aspect when executing the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method steps in any of the embodiments of the first aspect described above.

In a fifth aspect, the present application also provides a computer program product comprising a computer program that, when executed by a processor, performs the method steps in any of the embodiments of the first aspect described above.

According to the map merging method, the map merging device, the robot, the storage medium and the program product, a plurality of local point cloud maps in a target area are obtained, relative pose transformation between initial nodes in each adjacent local point cloud map is determined according to a superposition area of each local point cloud map and node information in each local point cloud map, and coordinate alignment transformation is carried out on all node information in each local point cloud map based on the relative pose transformation between the initial nodes in each adjacent local point cloud map, so that a global point cloud map of the target area is obtained. According to the method, a superposition area exists among local point cloud maps, each local point cloud map comprises node information of a robot in the process of building the local point cloud map, the relative pose among starting nodes in each adjacent local point cloud map can be directly obtained through a plurality of local point cloud maps, each local point cloud map can be converted to the same coordinate system according to the relative pose of the starting nodes, and the global point cloud map is obtained.

Drawings

FIG. 1 is a diagram of an exemplary map merging method;

FIG. 2 is a flow diagram illustrating a method for map merging in one embodiment;

FIG. 3 is a flow diagram illustrating a map merging method according to one embodiment;

FIG. 4 is a flow diagram illustrating a method for map merging in one embodiment;

FIG. 5 is a flow diagram illustrating a map merging method in one embodiment;

FIG. 6 is a flow diagram illustrating a method for map merging in one embodiment;

FIG. 7 is a flowchart illustrating a map merging method according to an embodiment;

FIG. 8 is a flowchart illustrating a map merging method according to one embodiment;

FIG. 9 is a flow diagram illustrating a map merging method in one embodiment;

FIG. 10 is a flow diagram illustrating a map merging method in accordance with one embodiment;

FIG. 11 is a flowchart illustrating a map merging method according to an embodiment;

FIG. 12 is a flow diagram illustrating a method for map merging in one embodiment;

fig. 13 is a block diagram showing the structure of the map merge device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The map merging method provided by the embodiment of the application can be applied to a robot, wherein the internal structure of the robot is shown in fig. 1. The memory in the internal structure of the robot includes a nonvolatile storage medium and an internal memory, the nonvolatile storage medium storing an operating system, a computer program, and a database; the internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database is used for storing a plurality of local point cloud map data and global point cloud map data. The network interface is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement the map merging method provided by the present application.

The following detailed description will specifically explain how the technical solutions of the present application and the technical solutions of the present application solve the above technical problems by embodiments and with reference to the accompanying drawings. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. In the map merging method provided by the present application, the execution subject may be a robot, or may be a map merging device, and the device may be implemented as a part or all of a robot by software, hardware, or a combination of software and hardware. It should be apparent that the embodiments described are some, but not all embodiments of the present application.

In one embodiment, as shown in fig. 2, a map merging method is provided, which is described by taking the robot in fig. 1 as an example, and includes the following steps:

s201, a plurality of local point cloud maps in a target area are obtained, overlapping areas exist among the local point cloud maps, and each local point cloud map comprises node information of a robot in the process of building the local point cloud map.

The robot comprises a robot body, a target area, a point cloud map and a plurality of target sub-areas, wherein the target area represents an area corresponding to the point cloud map needing to be constructed, when the target area is large, the target area can be divided into the plurality of target sub-areas, the point cloud map constructed by the robot in each target sub-area is used as a local point cloud map, and each target sub-area corresponds to one local point cloud map. The ranges of the multiple target sub-regions may be the same size, and the ranges of the corresponding local point cloud maps may be the same size, or the ranges of the multiple target sub-regions may be different. The surface shape of the target area can be any shape, and the local point cloud map corresponding to the target area can also be any shape. For example, the surface shape of the target area and the local point cloud map corresponding to the target area may be a triangle, a rectangle, a circle, or the like. In the process of dividing the target area, the target area can be divided equally, or the target area can be divided according to actual requirements. The node information of the robot comprises node poses, loop constraint relations among the nodes, point cloud descriptors of the nodes and the like. In the process of constructing a local point cloud map corresponding to a certain target sub-area through a robot, the robot moves in the target sub-area, and the local point cloud map of the target sub-area is generated through a simultaneous localization and mapping (SLAM) system of the robot.

Optionally, the robot may divide the target area into a plurality of sub-areas, and construct a plurality of local point cloud maps corresponding to the plurality of target sub-areas in real time by moving in the plurality of target sub-areas, so as to obtain a plurality of local point cloud maps in the target area. Optionally, the robot may also match the identification information of the target area with any one of the identification information in the point cloud map library, and if the matching is successful, obtain a plurality of pre-constructed local point cloud maps from the point cloud map library according to the identification information, so as to obtain a plurality of local point cloud maps in the target area. The point cloud map library comprises a plurality of pre-constructed local point cloud maps and corresponding relations between the local point cloud maps and the identification information. The identification information includes keyword information and storage time information of a plurality of point cloud maps of the target area.

Further, it can be understood that, since a plurality of adjacent target sub-regions exist in the target region, and the adjacent target sub-regions share the same edge, when the position of the robot is within a preset distance range of the common edge in the process of constructing the local point cloud maps corresponding to the adjacent target sub-regions, an overlapping region exists between two local point cloud maps constructed by the SLAM system in the robot.

S202, determining relative pose transformation between initial nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map.

The starting node is the starting node of each local point cloud map at the starting position of each target sub-area when the point cloud map is constructed for each target sub-area. The pose comprises a position and a pose, the position refers to a three-dimensional coordinate position of the robot in a current world map, and the pose refers to a three-dimensional rotation angle of the robot. The relative pose transformation refers to the transformation between the position and the posture between two nodes, and the relative pose transformation of the starting node refers to the transformation between the position and the posture of the starting position of two adjacent local point cloud maps.

Specifically, the robot can determine an initial node of each local point cloud map and a node pose of the initial node according to node information of each local point cloud map, then determine a plurality of adjacent local point cloud maps according to a superposition area of each local point cloud map, and for each pair of adjacent local point cloud maps, convert the node poses of the initial nodes in the two adjacent local point cloud maps to the same world coordinate system to obtain relative pose conversion of the node poses of the initial nodes in the two adjacent local point cloud maps in the same world coordinate system.

And S203, performing coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the initial nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area.

Specifically, the robot can perform coordinate alignment conversion on all node information in the adjacent local point cloud maps according to relative pose transformation between initial nodes in the adjacent local point cloud maps, the adjacent local point cloud maps are converted to the same coordinate system, and the two adjacent local point cloud maps are fused into a point cloud map. And fusing the fused point cloud map and the adjacent local point cloud map according to the method, so as to fuse all the local point cloud maps into a complete point cloud map, and optimizing the position and orientation map of the complete point cloud map to obtain a global point cloud map of the target area.

It can be understood that the method not only can complete the merging of the point cloud maps, but also can merge a plurality of 2D grid maps and merge the grid maps into a global grid map.

In the map merging method, a plurality of local point cloud maps in a target area are obtained, relative pose transformation between initial nodes in each adjacent local point cloud map is determined according to a superposition area of each local point cloud map and node information in each local point cloud map, and coordinate alignment conversion is carried out on all node information in each local point cloud map based on the relative pose transformation between the initial nodes in each adjacent local point cloud map, so that a global point cloud map of the target area is obtained. According to the method, the overlap area exists between the local point cloud maps, each local point cloud map comprises node information of a robot in the process of constructing the local point cloud map, the relative pose between the initial nodes in each adjacent local point cloud map can be directly obtained through the local point cloud maps, each local point cloud map can be converted to the same coordinate system according to the relative pose of the initial nodes, the global point cloud map is obtained, the process only involves calculation and processing of point cloud map data, the prior art involves calculation and processing of the point cloud map and GPS positioning data, compared with the prior art, the method is small in data quantity and data dimension, simple in calculation process, and capable of reducing complexity of combining the local point cloud maps into the global point cloud map.

Fig. 3 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for determining relative pose transformation between starting nodes in each adjacent local point cloud map according to the overlapping area of each local point cloud map and the node information in each local point cloud map. On the basis of the embodiment shown in fig. 2, as shown in fig. 3, the method may include the following steps:

s301, determining all adjacent local point cloud maps in each local point cloud map according to the overlapping area of each local point cloud map.

Specifically, the robot may number the overlapping area of each local point cloud map, determine two local point cloud maps corresponding to each number, and obtain all adjacent local point cloud maps in the plurality of local point cloud maps. Wherein, each local point cloud map may have one or more adjacent local point cloud maps. For example, if the length of the target area is 8 meters and the width is 2 meters, and the target area needs to be divided into 4 target sub-areas, the target area may be divided into 4 target sub-areas with the length of 2 meters and the width of 2 meters, or the target area may be divided into 4 target sub-areas with the length of 4 meters and the width of 1 meter. When the length of each target sub-area is 2 meters and the width of each target sub-area is 2 meters, two target sub-areas on the most edges of the 4 target sub-areas have an adjacent local point cloud map, and two target sub-areas in the middle have two adjacent local point cloud maps; when the length of the target sub-area is 4 meters and the width of the target sub-area is 1 meter, two adjacent local point cloud maps exist in each target sub-area.

S302, according to the node information in each local point cloud map, a plurality of matching node pairs in the adjacent local point cloud map are obtained.

The matching node pair refers to two nodes of which the matching degree of node information of the nodes in the two local point cloud maps is greater than a preset threshold value.

Specifically, each local point cloud map has a plurality of nodes, each node includes node information, and a one-to-one mapping relationship exists between the nodes and the node information. The robot can divide an adjacent local point cloud map into a first local point cloud map and a second local point cloud map, match node information of nodes in the first local point cloud map with node information of all nodes in the second local point cloud map according to a preset sequence, and determine two nodes in the adjacent local point cloud map as matching node pairs if the node information of the nodes in the first local point cloud map is successfully matched with the node information of any node in the second local point cloud map; and if the similarity of two nodes in the adjacent local point cloud map is greater than a preset threshold value, matching the node information of the node in the first local point cloud map with the node information of the next node in the second local point cloud map until the node information of the node in the first local point cloud map is matched with the node information of all the nodes in the second local point cloud map, and obtaining a plurality of matched node pairs in the adjacent local point cloud map. There may be multiple matching node pairs in each two adjacent local point cloud maps.

And S303, acquiring a node loop constraint relation between each matching node pair in each adjacent local point cloud map.

The loop constraint relation of the nodes refers to the difference of poses between the starting node and the ending node in the multi-frame key frame.

Specifically, the robot can perform point cloud registration on the adjacent local point cloud maps, and the adjacent local point cloud maps are converted into the same coordinate system according to the relative pose transformation between the matching node pairs to form a point cloud map. For any matching node pair in the adjacent local point cloud map, any matching node pair can be used as a starting node and an ending node of the adjacent local point cloud map, and the relative pose transformation of each matching node pair in the adjacent local point cloud map is determined as a node loop constraint relation.

S304, determining relative pose transformation between initial nodes in each adjacent local point cloud map according to the node loop constraint relation between each matching node pair in each adjacent local point cloud map.

Optionally, the robot may determine a node loop constraint relationship between any one of the matching node pairs in the adjacent local point cloud map as the relative pose transformation between the start nodes in the adjacent local point cloud map. Optionally, the robot may sort each matching node pair in the adjacent local point cloud map according to the matching value between the matching node pairs, and determine a node loop constraint relationship between the matching node pair with the highest matching value in the adjacent local point cloud map as the relative pose transformation between the starting nodes in the adjacent local point cloud map. In this embodiment, a manner of determining the relative pose transformation between the start nodes in each adjacent local point cloud map according to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map is not limited.

In the map merging method, all adjacent local point cloud maps in each local point cloud map are determined according to the overlapping area of each local point cloud map, a plurality of matching node pairs in the adjacent local point cloud maps are obtained according to node information in each local point cloud map, the node loop constraint relation between the matching node pairs in each adjacent local point cloud map is obtained, and the relative pose transformation between the initial nodes in each adjacent local point cloud map is determined according to the node loop constraint relation between the matching node pairs in each adjacent local point cloud map. According to the method, all adjacent local point cloud maps in each local point cloud map can be accurately determined according to the overlapping area of each local point cloud map, and for each pair of adjacent local point cloud maps, a plurality of matching node pairs in the adjacent point cloud maps can be accurately determined according to node information in the adjacent local point cloud maps, so that the relative pose transformation between initial nodes in each adjacent local point cloud map can be accurately determined according to the loop constraint relation between the matching node pairs, and the accuracy of obtaining the relative pose transformation between the initial nodes is improved.

Fig. 4 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to the technical scheme that if each adjacent local point cloud map comprises a first local point cloud map and a second local point cloud map; and the node information comprises point cloud descriptors of the nodes; an optional implementation manner of the multiple matching node pairs in the adjacent local point cloud map is obtained according to the node information in each local point cloud map. On the basis of the embodiment shown in fig. 3, as shown in fig. 4, the method may include the following steps:

s401, aiming at each adjacent local point cloud map, matching point cloud descriptors of all reference nodes in the second local point cloud map with point cloud descriptors of all loop candidate nodes in the first local point cloud map.

Specifically, the robot may perform processing such as identification, segmentation, resampling, registration surface reconstruction, and the like on the local point cloud map by extracting the point cloud descriptor. Each adjacent local point cloud map may be divided into a first local point cloud map and a second local point cloud map, each node in the first local point cloud map may be determined as a loop candidate node, and each node in the second local point cloud map may be determined as a reference node. The robot can calculate the similarity between the point cloud descriptors of the reference nodes and the point cloud descriptors of the loopback candidate nodes, and if the similarity is greater than a preset threshold value, the matching is successful; and if the similarity is smaller than the preset threshold, the matching is unsuccessful. The node information may be a point cloud descriptor, or other descriptors with high robustness.

S402, determining the reference nodes and the loopback candidate nodes successfully matched with the point cloud descriptors as matching node pairs to obtain a plurality of matching node pairs in each adjacent local point cloud map.

Specifically, the robot may match the point cloud descriptor of the reference node with the point cloud descriptor of any one of the loopback candidate nodes, and if the point cloud descriptor of the reference node is successfully matched with the point cloud descriptor of any one of the loopback candidate nodes, determine the reference node and the loopback candidate node as a matching node pair; and if the point cloud descriptor of the reference node is unsuccessfully matched with the loop candidate node, matching the reference node with the next node in each loop candidate node until the point cloud descriptor of each reference node in the second local point cloud map is matched with each loop candidate node in the first local point cloud map respectively, and obtaining a plurality of matched node pairs in the adjacent local point cloud map.

It can be understood that matching node pairs that are successfully matched have a certain probability that the robot reaches the same area, resulting in that point cloud descriptors of the matching node pairs are relatively similar, i.e., there may be a loop between the matching node pairs in the two maps.

In the map merging method, for each adjacent local point cloud map, point cloud descriptors of each reference node in the second local point cloud map are respectively matched with point cloud descriptors of each loopback candidate node in the first local point cloud map, the reference node and the loopback candidate node which are successfully matched with the point cloud descriptors are determined as matching node pairs, and a plurality of matching node pairs in each adjacent local point cloud map are obtained. According to the method, point cloud descriptors of nodes in two adjacent local point cloud maps are matched, the point cloud descriptors comprise a plurality of pieces of information, the plurality of pieces of information of the two nodes can be comprehensively matched, and the accuracy of the obtained matched node pairs is higher.

Fig. 5 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for obtaining a node loop constraint relation between matching node pairs in adjacent local point cloud maps. On the basis of the embodiment shown in fig. 4, as shown in fig. 5, the method may include the following steps:

s501, aiming at any matching node pair, a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matched with each loop candidate node are obtained.

Specifically, there may be multiple matching node pairs in two adjacent local point cloud maps, and for each matching node pair, the matching node pair includes a loopback candidate node and a reference node. Optionally, the robot may obtain a loopback candidate local point cloud map corresponding to the loopback candidate node from the point cloud database according to the identification information of the loopback candidate node; and acquiring a reference point cloud map corresponding to the reference node from the point cloud database according to the identification information of the reference node. Optionally, the robot may splice point cloud maps corresponding to multiple loop candidate frames in the loop candidate nodes to obtain loop candidate local point cloud maps corresponding to the loop candidate nodes; and splicing the reference point cloud maps corresponding to the plurality of reference frames in the reference nodes to obtain a reference local point cloud map corresponding to the reference nodes.

Optionally, fig. 6 is a schematic flowchart of a map merging method provided in the embodiment of the present application. The embodiment of the application relates to an optional implementation mode for obtaining a loop candidate local point cloud map corresponding to each loop candidate node in a matching node pair and a reference local point cloud map corresponding to a reference node matched with each loop candidate node. On the basis of the embodiment shown in fig. 5, as shown in fig. 6, the method may include the following steps:

s601, obtaining a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node.

Specifically, the reference frame corresponding to the reference node refers to a point cloud image acquired by the robot at the reference node, and the loop candidate frame corresponding to the loop candidate node refers to a point cloud image acquired by the robot at the loop candidate node. The robot may obtain the reference frame corresponding to each reference node from the database according to the identification information of the reference node, and the robot may obtain the loopback candidate frame corresponding to each loopback candidate node from the database according to the identification information of the loopback candidate node.

And S602, splicing the loop candidate frames and the key frames according to the poses of the key frames of the preset number adjacent to the front and back of each loop candidate frame to obtain a loop candidate local point cloud map corresponding to each loop candidate node.

Specifically, the loop candidate point cloud map is composed of point cloud maps corresponding to a plurality of loop candidate frames. For each loop candidate frame, the robot can determine a key frame adjacent to the loop candidate frame according to the generation time of the loop candidate frame, splice point cloud maps corresponding to the key frames and the loop candidate frames according to the poses of the key frames and the loop candidate frames, combine the point cloud maps corresponding to a plurality of frames into a large point cloud map, and determine the point cloud map as a loop candidate local point cloud map. And processing each loopback candidate frame by the method to obtain a loopback candidate local point cloud map corresponding to each loopback candidate node. For example, the front and rear 25 frames of the loop candidate frame may be determined as key frames to obtain 50 key frames, the 50 key frames and the point cloud map corresponding to the loop candidate frame are spliced to obtain a loop candidate local point cloud map, and the loop candidate local point cloud map is recorded as pointclosed 1.

And S603, converting each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference node matched with each loop candidate node.

Specifically, the reference local point cloud map is composed of point cloud maps corresponding to a plurality of reference frames. For each reference frame, the robot can convert the reference frame into a coordinate system corresponding to the loop candidate frame according to the pose of the loop candidate frame corresponding to the reference frame, perform splicing operation on the reference frame and the key frame according to the poses of a preset number of key frames adjacent to the reference frame in front and back, combine the point cloud maps corresponding to a plurality of frames into a large point cloud map, and determine the point cloud map as a reference local point cloud map. For example, the front and rear 25 frames of the reference frame may be determined as key frames to obtain 50 key frames, the 50 key frames and the point cloud map corresponding to the reference frame are spliced to obtain a reference local point cloud map, and the reference local point cloud map is recorded as pointclosed 2.

In the map merging method, a reference frame corresponding to each reference node and a loop candidate frame corresponding to each loop candidate node are obtained, the loop candidate frames and the key frames are spliced according to the poses of a preset number of key frames adjacent to the front and back of each loop candidate frame to obtain a loop candidate local point cloud map corresponding to each loop candidate node, and each reference frame is converted according to the poses of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference node matched with each loop candidate node. According to the method, a preset number of key frames adjacent to a reference frame and a loop candidate frame are spliced, so that a loop candidate local point cloud map can be accurately obtained; the reference frame is converted into the loop candidate frame, so that a reference local point cloud map can be accurately obtained; meanwhile, the reference node and the loop candidate node are positioned in the same world coordinate system, so that the loop candidate local point cloud map and the reference point cloud map can be more conveniently compared, and the obtained result is more accurate.

And S502, carrying out point cloud registration on the matched loop candidate local point cloud map and the reference local point cloud map.

The point cloud registration refers to converting two point sets so that the two point sets are located in the same coordinate to form a new point set.

Specifically, the robot can convert the loop candidate local point cloud map and the reference local point cloud map through a related point cloud registration algorithm, so that the loop candidate local point cloud map and the reference local point cloud map are in the same coordinate system to form a local point cloud map. For example, the related Point cloud registration algorithm includes Iterative Closest Point algorithm (ICP), Kernel Correlation (KC), Robust Point Matching (RPM), and the like. For example, point cloud registration is carried out on the local point cloud map pointcloud1 of the loop candidate and the reference local point cloud map pointcloud2, and pointcloud1 and pointcloud2 are fused into a point cloud map.

And S503, after the point cloud registration is completed, determining the relative pose between the loop candidate node and the reference node in each matching node pair as the node loop constraint relation between each matching node pair.

Specifically, after the point cloud registration of the loop candidate local point cloud map and the reference local point cloud map is completed in step S502, the loop candidate local point cloud map and the reference local point cloud map form a point cloud map, the loop candidate node is determined as the first node in the point cloud map, and the reference node is determined as the last node in the point cloud map, that is, the node loop constraint relationship in the point cloud map is the relative pose between the loop candidate node and the reference node.

It can be understood that the successful point cloud registration is considered that the similarity of the point cloud data of the reference node and the loop candidate node is high, that is, a loop exists between the matching node pairs in the two maps.

In the map merging method, for any matching node pair, a loop candidate local point cloud map corresponding to each loop candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matched with each loop candidate node are obtained, point cloud registration is performed on the matched loop candidate local point cloud map and the reference local point cloud map, and after the point cloud registration is completed, the relative pose between the loop candidate node and the reference node in each matching node pair is determined as the node loop constraint relation between each matching node pair. According to the method, point cloud registration is carried out on the local point cloud map and the reference point cloud map, on the basis that the robot is determined to reach the same area with a certain probability, the robot is further determined to reach the same area in the adjacent local point cloud map, the determining process of the node loop-back constraint relation is more accurate, and the obtained node loop-back constraint relation is more accurate.

Fig. 7 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for determining relative pose transformation between starting nodes in each adjacent local point cloud map according to a node loop constraint relation between each matching node pair in each adjacent local point cloud map. On the basis of the embodiment shown in fig. 3, as shown in fig. 7, the method may include the following steps:

s701, obtaining root mean square error values between all matching node pairs in each adjacent local point cloud map.

Specifically, the root mean square error value refers to the square root of the ratio of the square of the deviation between the predicted value and the actual value to the observation times. The robot can obtain the root mean square error value between the matching node pairs according to the node information of the matching node pairs in each adjacent local point cloud map and a related calculation formula. The root mean square error value between all the matching node pairs in each adjacent local point cloud map can be obtained through the method.

S702, obtaining the target matching node pair with the minimum root mean square error value.

Specifically, the robot may sort the matching node pairs according to the root mean square error values between the matching node pairs, rank the matching node pairs with the larger root mean square error values in front of the matching node pairs and rank the matching node pairs with the smaller root mean square error values in back of the matching node pairs, and determine the matching node pair corresponding to the last root mean square error value in the sorted root mean square error values as the target matching node pair.

S703, determining a node loop constraint relation between a reference node and a loop candidate node in the target matching node pair as relative pose transformation between initial nodes in each adjacent local point cloud map; and the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop-back candidate node in the target matching node pair is the starting node in the first local point cloud map.

Specifically, after point cloud registration is performed on a first local point cloud map and a second local point cloud, the first local point cloud map and the second local point cloud map are in the same coordinate system, the first local point cloud map and the second local point cloud map are combined into the same point cloud map, for the point cloud map, a reference node can be used as an initial node of the point cloud map, a loopback candidate node can be used as a termination node of the point cloud map, and a node loopback constraint relationship between the reference node and the loopback candidate node is relative pose transformation between initial nodes in adjacent local point cloud maps. The relative pose transformation between the initial nodes in each adjacent local point cloud map is determined through the method.

In the map merging method, the root mean square error value between all matching node pairs in each adjacent local point cloud map is obtained, the target matching node pair with the minimum root mean square error value is obtained, and the node loop constraint relation between the reference node in the target matching node pair and the loop candidate node is determined as the relative pose transformation between the initial nodes in each adjacent local point cloud map. In the method, a reference node in a target matching node pair is an initial node in a second local point cloud map, a loop candidate node in the target matching node pair is an initial node in a first local point cloud map, after point cloud registration is carried out on the two local point cloud maps, the first local point cloud map and the second local point cloud map are fused into a point cloud map, the reference node and the loop candidate node are respectively used as an initial node and a termination node of the point cloud map, and more accurate relative pose transformation between the initial nodes in the adjacent local point cloud maps can be obtained.

Fig. 8 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for obtaining candidate matching node pairs. On the basis of the embodiment shown in fig. 7, as shown in fig. 8, the method may include the following steps:

s801, obtaining a difference value between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair.

Specifically, after the root mean square error value of each matching node pair is obtained in step S701, the difference between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair is calculated, so as to obtain a plurality of differences.

S802, eliminating the matching node pairs with the difference values larger than the preset difference value to obtain candidate matching node pairs.

Specifically, when the difference between the root mean square error value of the matching node pair and the root mean square error value of the target matching node pair is greater than a preset difference, it is indicated that the relative pose transformation between the matching node pairs cannot be used as a constraint condition, the matching node pairs with the difference greater than the preset difference are filtered out, and the remaining matching node pairs are determined to be candidate matching node pairs.

In the map merging method, the difference value between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair is obtained, and the matching node pairs with the difference values larger than the preset difference value are removed to obtain candidate matching node pairs. According to the method, part of the matching node pairs which do not meet the root mean square error value in the plurality of matching node pairs are removed, the influence of the part of the matching node pairs on the merging process of the plurality of local point cloud maps can be avoided, and the efficiency of merging the local point cloud maps is improved.

Fig. 9 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for obtaining a global point cloud map of a target area by performing coordinate alignment conversion on all node information in each local point cloud map based on relative pose transformation between starting nodes in each adjacent local point cloud map. On the basis of the embodiment shown in fig. 8, as shown in fig. 9, the method may include the following steps:

and S901, converting all node information in each local point cloud map into the same coordinate system according to the relative pose transformation between the initial nodes in each adjacent local point cloud map to obtain an initial global point cloud map of a target area.

Specifically, for each adjacent local point cloud map, the robot may transform the relative pose between the initial nodes in the adjacent local point cloud maps, and convert all the node information in the other local point cloud map into the global coordinate system where the other local point cloud map is located, so that the adjacent local point cloud maps are located in the same coordinate system, and determine the point cloud map obtained after conversion as the initial global point cloud map of the target area.

And S902, optimizing the pose of the nodes in the initial global point cloud map through a preset optimizer to obtain a global point cloud map of the target area.

The optimizer is a tool for guiding each parameter of the objective function to update to a proper direction and an appropriate size in the deep learning back propagation process, so that the updated parameters enable the value of the objective function to approach the global minimum continuously.

Specifically, the robot may optimize a pose corresponding to each node in the initial global point cloud map by using a preset optimizer, optimize an error of the pose corresponding to each node to a preset range in the process of optimizing the pose corresponding to each node in the initial global point cloud map, output the pose corresponding to the optimized node, and construct the global point cloud map according to the optimized node pose and point cloud data corresponding to each node.

In the map merging method, according to the relative pose transformation between the initial nodes in each adjacent local point cloud map, all node information in each local point cloud map is converted into the same coordinate system to obtain an initial global point cloud map of a target area, and the poses of the nodes in the initial global point cloud map are optimized through a preset optimizer to obtain the global point cloud map of the target area. According to the method, all node information in each local point cloud map is converted into the same coordinate system, each local point cloud map is rapidly combined into an initial global map of a target area, and due to the fact that errors may exist among node poses in the initial global map, the poses of the nodes in the initial global point cloud map need to be optimized through a preset optimizer, and the accuracy of the obtained global point cloud map of the target area is higher.

Fig. 10 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for optimizing the pose of a node in an initial global point cloud map through a preset optimizer to obtain a global point cloud map of a target area. On the basis of the embodiment shown in fig. 9, as shown in fig. 10, the method may include the following steps:

s1001, inputting the node pose in the candidate point cloud map and the loop constraint relation into a preset optimizer, and optimizing the node pose through the optimizer to obtain a target node pose; the loop constraints comprise node loop constraint relations among the candidate matching node pairs, loop constraint relations in each local point cloud map and node loop constraint relations of adjacent nodes in each local point cloud map.

Wherein, using two adjacent local point cloud maps as an example, two adjacent local point cloud maps include first local point cloud map and second local point cloud map, and the loop constraint between two adjacent local point cloud maps includes: inter-frame constraint between adjacent nodes in the first local point cloud map, inter-frame constraint between adjacent nodes in the second local point cloud map, loop constraint in the first local point cloud map, loop constraint in the second local point cloud map, and loop constraint between the first local point cloud map and the second local point cloud map.

Specifically, the robot can take the node pose in the candidate point cloud map and the loopback constraint relation as the input of a preset optimizer, optimize the node pose in the candidate point cloud map through the preset optimizer, enable the node pose in the candidate point cloud map to meet a plurality of loopback constraint conditions, output the optimized node pose through the optimizer, and determine the optimized node pose as the target node pose.

S1002, constructing a global point cloud map of a target area based on the pose of the target node and the point cloud data corresponding to each node.

Optionally, the robot may determine the three-dimensional position of each target node in the candidate global point cloud map by a triangulation method, and construct the global point cloud map of the target area based on the target node pose, the point cloud data corresponding to each node, the three-dimensional positions of the plurality of target node points, the loop constraint relationship, and the like. Optionally, the robot may determine a re-projection constraint relationship of target nodes in the candidate global point cloud map by a re-projection method, and construct the global point cloud map of the target area based on the pose of the target nodes, the point cloud data corresponding to each node, the constraint relationship, and the re-projection constraint relationship.

In the map merging method, the node poses and the loop constraint relations in the candidate point cloud map are input into a preset optimizer, the node poses are optimized through the optimizer to obtain target node poses, and the global point cloud map of a target area is constructed based on the target node poses and point cloud data corresponding to each node. The loop constraints in the method comprise node loop constraint relations among candidate matching node pairs, loop constraint relations in each local point cloud map and node loop constraint relations of adjacent nodes in each local point cloud map.

Fig. 11 is a flowchart illustrating a map merging method according to an embodiment of the present application. The embodiment of the application relates to an optional implementation mode for obtaining a plurality of local point cloud maps in a target area. On the basis of the embodiment shown in fig. 2, as shown in fig. 11, the method may include the following steps:

s1101, constructing a pose graph corresponding to each initial local point cloud map according to the initial local point cloud maps of a plurality of areas in the target area.

Specifically, the initial local point cloud map is generated through a SLAM system of the robot, a driving path of the robot can be acquired according to the initial local point cloud map, and a pose map corresponding to the initial local point cloud map is constructed through the driving path. And constructing a pose map corresponding to the plurality of initial local point cloud maps by the method.

And S1102, transforming the point cloud data of each node to the global coordinate system of each initial local point cloud map through the node pose of each node in the pose map to obtain each local point cloud map.

Specifically, the point cloud data of each node is subjected to Euclidean transformation by using the node pose of each node in the pose graph, and the point cloud data of each node is transformed to the global coordinate system of each initial local point cloud map, so that a plurality of local point cloud maps are obtained. And simultaneously, storing node information of each local point cloud map and each attitude map node into a magnetic disk. For example, the first initial local point cloud map is transformed under the global coordinate system w0, and the second initial local point cloud map is transformed under the global coordinate system w 1.

In the map merging method, a pose graph corresponding to each initial local point cloud map is constructed according to the initial local point cloud maps of a plurality of areas in a target area, and point cloud data of each node is converted to a global coordinate system of each initial local point cloud map through node poses of each node in the pose graph to obtain each local point cloud map. The method can construct a pose graph according to each local point cloud map, and convert the point cloud data of each node into a global coordinate system by using the pose of each node, so that the local point cloud maps are conveniently spliced, and the efficiency of merging the local point cloud maps is improved.

In one embodiment, to facilitate understanding by those skilled in the art, the following describes a map merging method in detail, which may include, as shown in fig. 12:

s1201, constructing a pose graph corresponding to each initial local point cloud map according to the initial local point cloud maps of a plurality of areas in the target area;

s1202, point cloud data of each node is transformed to a global coordinate system of each initial local point cloud map through the node pose of each node in the pose map, and each local point cloud map is obtained;

s1203, determining all adjacent local point cloud maps in each local point cloud map according to the overlapping area of each local point cloud map;

s1204, aiming at each adjacent local point cloud map, respectively matching the point cloud descriptors of each reference node in the second local point cloud map with the point cloud descriptors of each loop candidate node in the first local point cloud map;

s1205, determining the reference node and the loopback candidate node which are successfully matched with the point cloud descriptor as matching node pairs to obtain a plurality of matching node pairs in each adjacent local point cloud map;

s1206, acquiring reference frames corresponding to the reference nodes and loop candidate frames corresponding to the loop candidate nodes;

s1207, splicing the loop candidate frames and the key frames according to the poses of the key frames of the preset number, which are adjacent to each other in front of and behind the loop candidate frames, so as to obtain a loop candidate local point cloud map corresponding to each loop candidate node;

s1208, converting each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference node matched with each loop candidate node;

s1209, point cloud registration is carried out on the matched loop candidate local point cloud map and the reference local point cloud map;

s1210, after point cloud registration is completed, determining the relative pose between the loop candidate node and the reference node in each matching node pair as the node loop constraint relation between each matching node pair;

s1211, obtaining root mean square error values between all matching node pairs in each adjacent local point cloud map;

s1212, obtaining a target matching node pair with the minimum root mean square error value;

s1213, determining a node loop constraint relation between a reference node and a loop candidate node in the target matching node pair as relative pose transformation between starting nodes in each adjacent local point cloud map;

s1214, obtaining a difference value between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

s1215, eliminating the matching node pairs with the difference values larger than the preset difference value to obtain candidate matching node pairs;

s1216, converting all node information in each local point cloud map into the same coordinate system according to the relative pose transformation between the initial nodes in each adjacent local point cloud map to obtain an initial global point cloud map of a target area;

s1217, inputting the node pose in the candidate point cloud map and the loop constraint relation into a preset optimizer, and optimizing the node pose through the optimizer to obtain a target node pose;

and S1218, constructing a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

It should be noted that, for the descriptions in S1201 to S1218, reference may be made to the relevant descriptions in the foregoing embodiments, and the effects thereof are similar, and details of this embodiment are not repeated herein.

Illustratively, taking two pre-constructed point cloud maps as an example, the two point cloud maps are respectively denoted as map0 and map1, and a certain overlapping area exists in map0 and map 1. The process of merging map0 and map1 comprises the following steps: (1) converting the poses of the nodes in the map0 and the map1 to the global coordinate system of the point cloud map by using Euclidean transformation, and converting the maps 0 and 1 to the w0 and w1 global coordinate systems respectively; (2) matching the point cloud descriptors of all the nodes in the map0 and the map1, if the matching is successful, determining the successfully matched node in the map0 as a current node, and determining the successfully matched node in the map1 as a loopback candidate node; (3) acquiring point clouds of front and rear 25 frames of key frames of the loopback candidate frame corresponding to the loopback candidate node in map1, and splicing 50 frames of key frames and the loopback candidate frame into a local point cloud map which is marked as pointclosed 1; converting the current frame into a pointclose 2 under a w0 coordinate system according to the poses of the current frame and the loopback candidate frame corresponding to the current node, performing point cloud registration on the pointclose 1 and the pointclose 2, and determining the relative pose between the current frame and the loopback candidate frame as loopback constraint if the point cloud registration is successful; (4) determining the loop constraint with the minimum root mean square error as the relative pose transformation between the map0 and the map1 starting point, and simultaneously rejecting the loop constraint which does not meet the preset condition in the loop constraints; (5) converting node poses of all nodes in map1 to a w0 coordinate system according to relative pose transformation between map0 and map1 starting points to obtain an initial global point cloud map; (6) inputting the node poses of all nodes in the initial global point cloud map and all constraint conditions into an optimizer, optimizing the node poses of all nodes through the optimizer, enabling the node poses of all nodes to meet all constraint conditions, obtaining optimized node poses, splicing point cloud data of the nodes according to the node poses of all nodes, and obtaining the global point cloud data after map0 and map1 are combined.

In the map merging method provided by this embodiment, there is an overlap area between local point cloud maps, each local point cloud map includes node information of a robot in a process of building the local point cloud map, a relative pose between starting nodes in each adjacent local point cloud map can be directly obtained through a plurality of local point cloud maps, each local point cloud map can be converted into the same coordinate system according to the relative pose of the starting nodes, and a global point cloud map is obtained.

It should be understood that, although the steps in the flowcharts related to the embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application further provides a map merging device for implementing the above related map merging method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the map merging device provided below can be referred to the limitations of the map merging method in the foregoing, and details are not described here.

In one embodiment, as shown in fig. 13, there is provided a map merging apparatus including: a first obtaining module 11, a determining module 12 and a converting module 13, wherein:

the system comprises a first acquisition module 11, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a plurality of local point cloud maps in a target area, a superposition area exists between the local point cloud maps, and each local point cloud map comprises node information of a robot in the process of constructing the local point cloud map;

the determining module 12 is configured to determine, according to the overlapping area of each local point cloud map and the node information in each local point cloud map, relative pose transformation between starting nodes in each adjacent local point cloud map;

and the conversion module 13 is configured to perform coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the start nodes in each adjacent local point cloud map, so as to obtain a global point cloud map of the target area.

The map merging device provided in this embodiment may implement the above method embodiments, and the implementation principle and technical effect are similar, which are not described herein again.

In one embodiment, the determining module includes: a first determining unit, a first obtaining unit and a second obtaining unit, wherein:

the first determining unit is used for determining all adjacent local point cloud maps in each local point cloud map according to the overlapping area of each local point cloud map;

the first acquisition unit is used for acquiring a plurality of matching node pairs in adjacent local point cloud maps according to node information in each local point cloud map;

the second acquisition unit is used for acquiring a node loop constraint relation between each matching node pair in each adjacent local point cloud map;

and the second determining unit is used for determining the relative pose transformation between the initial nodes in each adjacent local point cloud map according to the node loop constraint relationship between each matching node pair in each adjacent local point cloud map.

Optionally, the first obtaining unit is further configured to, when each adjacent local point cloud map includes a first local point cloud map and a second local point cloud map, and the node information includes a point cloud descriptor of a node, match, for each adjacent local point cloud map, the point cloud descriptor of each reference node in the second local point cloud map with the point cloud descriptor of each loop candidate node in the first local point cloud map; and determining the reference nodes and the loopback candidate nodes successfully matched with the point cloud descriptors as matching node pairs to obtain a plurality of matching node pairs in each adjacent local point cloud map.

Optionally, the second obtaining unit is further configured to obtain, for any matching node pair, a loopback candidate local point cloud map corresponding to each loopback candidate node in the matching node pair and a reference local point cloud map corresponding to a reference node matched with each loopback candidate node; carrying out point cloud registration on the matched loop candidate local point cloud map and the reference local point cloud map; and after the point cloud registration is completed, determining the relative pose between the loop candidate node and the reference node in each matching node pair as the node loop constraint relation between each matching node pair.

Optionally, the second obtaining unit is further configured to obtain a reference frame corresponding to each reference node and a loopback candidate frame corresponding to each loopback candidate node; splicing the loopback candidate frames and the key frames according to the poses of a preset number of key frames adjacent to each loopback candidate frame in front and back to obtain a loopback candidate local point cloud map corresponding to each loopback candidate node; and converting each reference frame according to the pose of each loop candidate frame to obtain a reference local point cloud map corresponding to the reference nodes matched with each loop candidate node.

Optionally, the second determining unit is further configured to obtain a root mean square error value between all matching node pairs in each adjacent local point cloud map; obtaining a target matching node pair with the minimum root mean square error value; determining a node loop constraint relation between a reference node and a loop candidate node in a target matching node pair as relative pose transformation between initial nodes in each adjacent local point cloud map; and the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop-back candidate node in the target matching node pair is the starting node in the first local point cloud map.

In one embodiment, the map merging apparatus further includes: second acquisition module and rejection module, wherein:

the second acquisition module is used for acquiring the difference value between the root mean square error value of each matching node pair and the root mean square error value of the target matching node pair;

and the eliminating module is used for eliminating the matching node pairs with the difference values larger than the preset difference value to obtain candidate matching node pairs.

In one embodiment, the conversion module further includes: an optimization unit and a construction unit, wherein:

the optimization unit is used for inputting the node pose in the candidate point cloud map and the loop constraint relation into a preset optimizer, and optimizing the node pose through the optimizer to obtain a target node pose; the loop constraint comprises a node loop constraint relation between candidate matching node pairs, a loop constraint relation in each local point cloud map and a node loop constraint relation of adjacent nodes in each local point cloud map;

and the construction unit is used for constructing a global point cloud map of the target area according to the target node pose and the point cloud data corresponding to each node.

Optionally, the constructing unit is further configured to construct a pose graph corresponding to each initial local point cloud map according to the initial local point cloud maps of the plurality of regions in the target region; and transforming the point cloud data of each node to the global coordinate system of each initial local point cloud map through the node pose of each node in the pose map to obtain each local point cloud map.

The map merging device provided in this embodiment may implement the method embodiments, and the implementation principle and technical effect are similar, which are not described herein again.

The various modules in the above-described assembly apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent of a processor in the robot, and can also be stored in a memory in the robot in a software form, so that the processor can call and execute operations corresponding to the modules.

In an embodiment, a robot is provided, comprising a memory in which a computer program is stored and a processor which, when executing the computer program, implements all of the above described method embodiments.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, carries out all the above-mentioned method embodiments.

In an embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, implements all of the above described method embodiments.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A map merging method applied to a robot is characterized by comprising the following steps:

acquiring a plurality of local point cloud maps in a target area, wherein a superposition area exists between the local point cloud maps, and each local point cloud map comprises node information in a constructed local point cloud map;

and performing coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation relation between the initial nodes in each adjacent local point cloud map to obtain the global point cloud map of the target area.

2. The method of claim 1, wherein determining a relative pose transformation relationship between start nodes in each of the adjacent local point cloud maps according to a coincidence region of each of the local point cloud maps and node information in each of the local point cloud maps comprises:

acquiring matching node pairs in the adjacent local point cloud maps according to node information in each local point cloud map;

3. The method of claim 2, wherein if each adjacent local point cloud map comprises a first local point cloud map and a second local point cloud map; and the node information in each local point cloud map comprises point cloud descriptors of nodes;

then, the obtaining of the matching node pairs in the adjacent local point cloud maps according to the node information in each local point cloud map includes:

for each adjacent local point cloud map, respectively matching the point cloud descriptors of the reference nodes in the second local point cloud map with the point cloud descriptors of the loopback candidate nodes in the first local point cloud map;

4. The method of claim 3, wherein obtaining a node loopback constraint relationship between each pair of matching nodes in each of the neighboring local point cloud maps comprises:

5. The method according to claim 4, wherein the obtaining of the loopback candidate local point cloud map corresponding to each loopback candidate node in the pair of matching nodes and the reference local point cloud map corresponding to the reference node matched with each loopback candidate node comprises:

acquiring a reference frame corresponding to each reference node and a loopback candidate frame corresponding to each loopback candidate node;

and converting the reference frames according to the poses of the loopback candidate frames to obtain a reference local point cloud map corresponding to the reference nodes matched with the loopback candidate nodes.

6. The method of any one of claims 3-5, wherein determining the relative pose transformation between the start nodes in each of the neighboring local point cloud maps according to the node loop constraint relationship between each pair of matching nodes in each of the neighboring local point cloud maps comprises:

determining a node loop-back constraint relationship between a reference node and a loop-back candidate node in the target matching node pair as a relative pose transformation between starting nodes in each adjacent local point cloud map; and the reference node in the target matching node pair is the starting node in the second local point cloud map, and the loop candidate node in the target matching node pair is the starting node in the first local point cloud map.

7. The method of claim 6, further comprising:

8. The method of claim 7, wherein the performing coordinate alignment transformation on all node information in each local point cloud map based on relative pose transformation between starting nodes in each adjacent local point cloud map to obtain a global point cloud map of the target area comprises:

converting all node information in each local point cloud map into the same coordinate system according to relative pose transformation between initial nodes in each adjacent local point cloud map to obtain an initial global point cloud map of the target area;

9. The method of claim 8, wherein the optimizing the pose of the node in the initial global point cloud map by a preset optimizer to obtain the global point cloud map of the target area comprises:

inputting the node pose and the loop constraint relation in the candidate point cloud map into a preset optimizer, and optimizing the node pose through the optimizer to obtain a target node pose; the loopback constraints comprise node loopback constraint relations among the candidate matching node pairs, loopback constraint relations in each local point cloud map and node loopback constraint relations of adjacent nodes in each local point cloud map;

and constructing a global point cloud map of the target area based on the target node pose and the point cloud data corresponding to each node.

10. The method of any one of claims 1-5, wherein obtaining a plurality of local point cloud maps in a target area comprises:

constructing a pose map corresponding to each initial local point cloud map according to the initial local point cloud maps of a plurality of areas in the target area;

and transforming the point cloud data of each node into a global coordinate system of each initial local point cloud map according to the node pose of each node in the pose map to obtain each local point cloud map.

11. A map merging apparatus, characterized in that the apparatus comprises:

and the conversion module is used for carrying out coordinate alignment conversion on all node information in each local point cloud map based on the relative pose transformation between the initial nodes in each adjacent local point cloud map to obtain the global point cloud map of the target area.

12. A robot comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any of claims 1 to 10 when executing the computer program.