CN114528453A

CN114528453A - Global repositioning method and device, electronic equipment and storage medium

Info

Publication number: CN114528453A
Application number: CN202111676621.8A
Authority: CN
Inventors: 王璀
Original assignee: Beijing Yihang Yuanzhi Technology Co Ltd
Current assignee: Beijing Yihang Yuanzhi Technology Co Ltd
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2022-05-24

Abstract

The application relates to a global relocation method, a global relocation device, an electronic device and a storage medium. The global relocation method comprises the following steps: determining a target semantic map and a semantic map to be matched of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the semantic map to be matched comprises a plurality of semantic targets to be matched; determining a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, wherein a node of the minimum spanning tree is the semantic target to be matched; constructing a search tree based on the target semantic map, wherein the nodes of the search tree are the standard semantic targets; determining a growth tree matched with the minimum spanning tree from the constructed search tree; and determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree.

Description

Global repositioning method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a global relocation method and apparatus, an electronic device, and a storage medium.

Background

With the continuous development of positioning technology and automatic control technology, many intelligent devices have realized autonomous movement functions, ranging from large to unmanned vehicles, ranging from small to cleaning robots, and the like. Generally, when the intelligent device operates in a target area, the position of the intelligent device in a map of the target area can be determined by means of sensor information, and autonomous navigation can be achieved by combining related information in the map.

The map of the target area can be generally constructed as a visual feature map, a laser point cloud map, a probability grid map, a semantic map, and the like. The semantic map is an enhanced representation of the environment, contains geometric information and high-level qualitative features, and belongs to high-level features. The semantic object is influenced by environmental factors such as illumination, seasons and the like to the lowest extent, and the robustness is higher. Moreover, the semantic features have high information content and sparse distribution, thereby greatly simplifying the information expression of the map and reducing the capacity required by the map. Therefore, the semantic map is relatively more suitable for large-scale positioning and mapping tasks, such as high-speed positioning navigation, autonomous parking and other fields.

Errors may exist in sensor information of the intelligent device, and due to accumulation of the errors, positioning information is misaligned, so that navigation accuracy is affected, and therefore the pose needs to be determined again. In addition, after the intelligent device is started or a kidnapping occurs, the pose of the intelligent device can be difficult to determine in a new environment. Therefore, how to improve the global positioning efficiency of the intelligent device becomes an important problem of improving the navigation accuracy at present.

Disclosure of Invention

In order to solve the technical problem, the present application provides a global relocation method, device, electronic device, and storage medium. The global positioning efficiency of the intelligent device is improved, and the navigation precision is further improved.

In a first aspect, the present application provides a global relocation method, including:

determining a target semantic map and a semantic map to be matched of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the semantic map to be matched comprises a plurality of semantic targets to be matched;

determining a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, wherein a node of the minimum spanning tree is the semantic target to be matched;

constructing a search tree based on the target semantic map, wherein nodes of the search tree are the standard semantic targets;

determining a growth tree matched with the minimum spanning tree from the constructed search tree;

and determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree.

Optionally, the determining a minimum spanning tree based on a complete graph based on the semantic map to be matched, where a node of the minimum spanning tree is the semantic target to be matched, includes:

aiming at each semantic target to be matched, taking the semantic target to be matched as a node, and constructing an undirected complete graph;

aiming at any two semantic objects to be matched, determining the weight of an edge between nodes constructed by the two semantic objects to be matched according to translation information of the two semantic objects to be matched;

determining a minimum spanning tree of the undirected full graph based on weights of edges in the undirected full graph.

Optionally, the determining a growth tree matching the minimum spanning tree from the constructed search tree includes:

constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree;

and determining a growth tree from the constructed search tree based on the survival rule tree, wherein the growth tree meets the survival rule represented by the survival rule tree.

Optionally, the survival rule corresponding to each branch includes: the relative pose and the pre-estimated error variance of the node pair corresponding to each branch;

constructing a rule-of-survival tree based on a rule-of-survival corresponding to each branch in the minimum spanning tree, comprising:

aiming at the node pair corresponding to each branch in the minimum spanning tree, calculating the relative pose corresponding to the node pair;

calculating the pre-estimated error variance corresponding to each node pair aiming at the node pair corresponding to each branch in the minimum spanning tree;

generating a survival rule corresponding to each branch by using the position of each branch in the minimum spanning tree, the relative pose of the node pair corresponding to the branch and the estimated error variance;

and determining a survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

Optionally, the survival rule corresponding to each branch further includes: attribute characteristics of each node;

determining attribute characteristics of each node corresponding to each branch in the minimum spanning tree;

generating survival rules corresponding to the branches by using the position of each branch in the minimum spanning tree, the relative pose of the node pair corresponding to the branch, the estimated error variance and the attribute characteristics of each node;

Optionally, the calculating, for the node pair corresponding to each branch in the minimum spanning tree, a pre-estimation error variance corresponding to the node pair includes:

determining an observation error and a distance error corresponding to the target equipment;

determining the distance between the targets to be matched corresponding to the node pairs aiming at the node pairs corresponding to each branch in the minimum spanning tree;

and determining the pre-estimated error variance corresponding to the node pair according to the observation error, the distance error and the distance corresponding to the node pair.

Optionally, determining a growth tree from the constructed search tree based on the rule-of-survival tree, where the growth tree satisfies the rule-of-survival represented by the rule-of-survival tree, includes:

selecting a node from the minimum spanning tree as a tree root node;

selecting at least one standard semantic target which has the same attribute characteristics as the tree root nodes from the target semantic map as a seed of a growth tree;

traversing each node of the minimum spanning tree upward starting from the tree root node;

for each node of the minimum spanning tree, performing the following steps:

determining branches to be grown corresponding to the nodes and branch nodes forming the branches to be grown with the nodes;

according to the survival rule tree, searching a standard semantic target matched with the branch nodes from the search tree to serve as growth nodes of the growth tree;

and after traversing the minimum spanning tree, determining a growing tree according to the seeds and the corresponding growing nodes.

Optionally, the selecting a node from the minimum spanning tree as a tree root node includes:

for each node in the minimum spanning tree, determining attribute characteristics of the node;

determining, for each attribute feature of the node, a first number corresponding to nodes in the minimum spanning tree having the same value as the attribute feature of the node;

determining, for each attribute feature of the node, a second number corresponding to a standard semantic object in the object semantic map having a same value as the attribute feature value of the node;

determining a specificity score of the node according to the first quantity and the second quantity corresponding to each attribute feature of the node;

and selecting the node with the highest specificity score as a tree root node.

Optionally, the searching for the standard semantic object matched with the branch node from the search tree as the growth node of the growth tree according to the rule-of-survival tree includes:

calculating a circumscribed cube of the minimum spanning tree;

calculating the distance from the tree root node to each vertex of the cube;

taking the maximum distance from the tree root node to each vertex of the cube as a search radius;

for each seed, determining a corresponding sub search tree from the search trees based on the search radius;

and aiming at each node of the minimum spanning tree, searching a standard semantic target matched with the node from each sub-search tree as a growth node of the growth tree according to a survival rule corresponding to the node.

aiming at the sub search tree corresponding to each seed, the following steps are executed:

determining a survival rule of the branches to be grown from the survival rule tree;

determining a current node in the sub-lookup tree;

searching nodes positioned in the searching radius of the current node of the searching tree in the sub searching tree as a first candidate node set;

determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown;

taking the final candidate node as a growth node of the growth tree;

and if the final candidate node is not determined, deleting the growth tree corresponding to the seed.

Optionally, the determining the survival rule of the branch to be grown from the survival rule tree includes:

determining the relative pose and the estimated error variance of the node pair corresponding to the branch to be grown from the survival rule tree;

determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown, wherein the determining comprises the following steps:

for each node in the first set of candidate nodes, determining a relative pose of the node and a current node of the sub-lookup tree;

determining whether the node is matched with the branch node or not according to the relative pose of the node and the current node of the sub search tree, the relative pose of the node pair corresponding to the branch to be grown and the estimated error variance;

and if the node is matched with the branch node, determining the node as a final candidate node.

Optionally, determining the survival rule of the branch to be grown from the survival rule tree includes:

determining the relative pose of the node pair corresponding to the branch to be grown, the estimated error variance and the attribute characteristics of the branch node from the survival rule tree;

determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown, including:

for each node in the first set of candidate nodes, determining whether the attribute characteristics of the node and the attribute characteristics of the branch node are the same;

if the attribute characteristics of the nodes are different from the attribute characteristics of the branch nodes, deleting the nodes from the first candidate node set to obtain a second candidate node set;

for each node in the second set of candidate nodes, determining a relative pose of the node and a current node of the child look-up tree;

Optionally, the determining whether the node is matched with the branch node according to the relative pose of the node and the current node of the sub search tree, the relative pose of the node pair corresponding to the branch to be grown, and the pre-estimated error variance includes:

determining a mismatch error according to the relative poses of the node and the current node of the sub search tree;

determining an actual error variance according to the mismatch error;

and determining whether the node is matched with the branch node or not according to the actual error variance and the estimated error variance and by combining chi-square test.

In a second aspect, the present application provides a global relocation apparatus, including:

the semantic target determining module is used for determining a target semantic map and a semantic map to be matched of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the semantic map to be matched comprises a plurality of semantic targets to be matched;

the minimum spanning tree determining module is used for determining a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, and a node of the minimum spanning tree is the semantic target to be matched;

the search tree determining module is used for constructing a search tree based on the target semantic map, and the nodes of the search tree are the standard semantic targets;

the growth tree determining module is used for determining a growth tree matched with the minimum generation tree from the constructed search tree;

and the pose determining module is used for determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree.

Optionally, the minimum spanning tree determining module is specifically configured to:

determining a minimum spanning tree of the undirected full graph based on the weights of the edges in the undirected full graph.

Optionally, the growth tree determining module is specifically configured to:

the spanning tree determining module is specifically configured to, when constructing a rule-of-survival tree based on a rule-of-survival corresponding to each branch in the minimum spanning tree:

Optionally, when the spanning tree determining module calculates the estimated error variance corresponding to the node pair for the node pair corresponding to each branch in the minimum spanning tree, the growing tree determining module is specifically configured to:

aiming at the node pairs corresponding to each branch in the minimum spanning tree, determining the distance between the targets to be matched corresponding to the node pairs;

Optionally, the growing tree determining module is configured to determine a growing tree from the constructed search tree based on the rule-of-survival tree, where the growing tree satisfies a rule of survival represented by the rule-of-survival tree, and is specifically configured to:

selecting a node from the minimum spanning tree as a tree root node;

for each node of the minimum spanning tree, performing the following steps:

Optionally, when the spanning tree determining module selects one node from the minimum spanning tree as a tree root node, the spanning tree determining module is specifically configured to:

determining, for each attribute feature of the node, a second number in the target semantic map corresponding to a standard semantic target having the same attribute feature value as the node;

and selecting the node with the highest specificity score as a tree root node.

Optionally, when the standard semantic object matched with the branch node is searched from the search tree as the growth node of the growth tree according to the rule of survival tree, the growth tree determining module is specifically configured to:

calculating a circumscribed cube of the minimum spanning tree;

calculating the distance from the tree root node to each vertex of the cube;

determining a current node in the sub-lookup tree;

taking the final candidate node as a growth node of the growth tree;

Optionally, when determining the survival rule of the branch to be grown from the survival rule tree, the growing tree determining module is specifically configured to:

the growing tree determining module is specifically configured to, when determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown:

Optionally, the growing tree determining module is specifically configured to, when determining the survival rule of the branch to be grown from the rule of survival tree:

determining the relative pose of the node pair corresponding to the branch to be grown, the estimated error variance and the attribute characteristics of the branch nodes from the survival rule tree;

the growing tree determining module is specifically configured to, when determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown,:

Optionally, when the growing tree determining module determines whether the node matches the branch node according to the relative pose of the node and the current node of the sub search tree, the relative pose of the node pair corresponding to the branch to be grown, and the prediction error variance, the growing tree determining module is specifically configured to:

determining an actual error variance according to the mismatch error;

In a third aspect, the present application provides an electronic device, comprising: a memory and a processor;

the memory to store program instructions;

the processor is configured to call and execute the program instructions in the memory to perform the method according to any one of the first aspect.

In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored therein; which when executed by a processor implements the method according to the first aspect.

In a fifth aspect, the present application provides a computer program product comprising: a computer program; which when executed by a processor implements the method according to the first aspect.

The application provides a global relocation method, a global relocation device, electronic equipment and a storage medium. The global relocation method comprises the following steps: determining a target semantic map and a semantic map to be matched of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the semantic map to be matched comprises a plurality of semantic targets to be matched; determining a minimum spanning tree based on an undirected complete graph based on the semantic map to be matched, wherein a node of the minimum spanning tree is the semantic target to be matched; constructing a search tree based on the target semantic map, wherein the nodes of the search tree are the standard semantic targets; determining a growth tree matched with the minimum spanning tree from the constructed search tree; and determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree. The problem of target matching is converted into the problem of growth tree growth, and the growth of the growth tree can select the optimal node based on all standard semantic targets in the target semantic map and grow into the growth tree which is optimally matched with the minimum generation tree. The target matching can be carried out based on all semantic targets in the target semantic map, so that the global relocation is really realized, and the global positioning efficiency of the intelligent equipment is improved.

Drawings

Fig. 1 is a schematic view of an application scenario provided in the present application.

Fig. 2 is a flowchart of a global relocation method according to an embodiment of the present application.

Fig. 3 is a flowchart of determining a growth node according to an embodiment of the present application.

Fig. 4 is a structural diagram of a growing tree according to an embodiment of the present application.

Fig. 5 is a schematic structural diagram of a global relocation apparatus according to an embodiment of the present application.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship, unless otherwise specified.

The embodiments of the present application will be described in further detail with reference to the drawings attached hereto.

The prime symbol convention in this application is as follows:

key symbols:

for object semantic maps (maps)

Represents;

the set (landmark sets) of semantic objects to be matched in the semantic map to be matched is used

Represents;

the pose quantity in the pose representation is an Orthogonal Matrix (Orthogonal Matrix) used for representing the pose

Represents;

and can also represent lie algebra represented by the rotation; for displacement/Translation Vector (transformation Vector)

And (4) showing.

Subscripts:

subscripts are used herein generally as an index or code to denote an individual, such as where the target element individuals are each denoted by a subscript, unless otherwise specified. Is shaped as

To represent

In (1)

The elements are selected from the group consisting of,

to represent

In

The elements are selected from the group consisting of,

is represented by

Produced by

。

And (3) labeling:

the superscript is used in the present invention to distinguish between attribution or classification in general, unless otherwise specified. For example, shaped as

Is represented by a category

Lower and

related amount

。

The above convention applies to most of the symbolic representations in the embodiments of the present application, and is not actually meant in a certain sense, and the formal explanations are merely for convenience of understanding, and different notations are all explained in the embodiments of the present application.

The semantic objects in the semantic map are influenced to the lowest degree by the environmental factors such as illumination, seasons and the like, so that the robustness is higher. Moreover, the semantic features contain high information quantity and are distributed sparsely, so that the information expression of the map can be greatly simplified, and the capacity required by the map is reduced. Therefore, the semantic map is distinct from a plurality of map types, and is more suitable for large-scale positioning and mapping tasks, such as the fields of high-speed positioning navigation, autonomous parking and the like.

The existing global matching and repositioning technology based on semantic objects has the following problems.

First, existing semantic map-based relocation techniques require a certain initial location state as a condition. For example, the relocation of outdoor high-precision maps depends on Real-time kinematic (RTK)/Global Positioning System (GPS) to give an initial state, and then local search matching is performed to perform relocation; semantic segmentation of an indoor parking lot/relocation of a target level map requires reliance on Ultra Wide Band (UWB) location technology/an initial reference parking point.

Second, the amount of information of semantic objects used, such as triangulation, is small. The information of category, direction, etc. cannot be covered.

Third, the number of features used is small. Often only a few frames of observed objects can be used.

Fourth, the coverage of semantic features for matching is small. If the semantic features of a large range are contained, the links of the geometric constraint can not pass the verification.

Fifth, a descriptor is required. The method is often used for clustering and similarity judgment, is strongly related to the environment or description mode, and the description mode needs to occupy additional memory and computing resources.

Sixth, the target matching can only process the object association and tracking for a short time, and it is difficult to provide an associated target solution set at a global level.

In order to solve the problems in the prior art, the application provides a global repositioning method, which aims to structure semantic features in a target map, match the semantic features with the semantic features in a map to be matched and complete repositioning. The invention can support a large number of structured semantic objects without giving initial states and quantized feature descriptors, and is not completely matched rigidly.

The global relocation method provided by the application can be applied to the aspects of data association, starting positioning of the intelligent equipment, loop detection, relocation after abnormal positioning and the like. Correspondingly, the target semantic map can be a complete semantic map which is already established, and the semantic map to be matched can be a semantic map which needs to be established at the moment of repositioning. The current pose can be repositioned by matching the map to be matched with the target semantic map.

Fig. 1 is a schematic view of an application scenario provided by the present application, and illustrates a process of relocation of an intelligent device by applying the method of the present application, by taking the above boot positioning scenario as an example. As shown in fig. 1, the household intelligent sweeping robot 101 is powered off when the electric quantity is exhausted in a certain sweeping process, and needs to reposition the current pose after charging and powering on, so as to determine the current position, and then plan the sweeping route again. After the robot is started, a current image (semantic map to be matched) is obtained, and semantic targets in the two maps are matched by using the global positioning method in combination with a pre-constructed map (target semantic map) of an indoor area so as to determine the current pose of the robot. In this scenario, objects such as furniture in a room may be semantic objects in the map.

The following embodiments may be referred to in the specific implementation manner of matching and repositioning the current pose through the map to be matched and the target semantic map.

Fig. 2 is a flowchart of a global relocation method according to an embodiment of the present application. The method of the embodiment can be applied to intelligent devices (target devices) which need to be relocated, such as unmanned vehicles, unmanned planes, household intelligent sweeping robots in the above scenes, and the like. The method of the embodiment comprises the following steps:

s201, determining a target semantic map and a to-be-matched semantic map of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the to-be-matched semantic map comprises a plurality of to-be-matched semantic targets.

When the pose of the target device is determined by means of feature matching, a semantic map of the current position of the target device (referred to as a semantic map to be matched in the present application) and a semantic map (referred to as a target semantic map in the present application) relatively standard (capable of accurately representing the real environment) in the current environment are required.

The target device can acquire semantic map information of the current position through a sensor (a depth camera, a laser radar and the like) equipped in the target device. And taking the marks or objects in the surrounding environment as semantic targets (referred to as semantic targets to be matched in the application) in the semantic map to be matched.

In different application scenes, the selection of the target semantic map is different. For example, in the boot positioning scenario described in fig. 1, a constructed accurate map of the environment where the target device is located is generally already available, and the constructed accurate map may be directly obtained as the target semantic map. Loop detection is generally applied to pose adjustment in a continuous positioning process to eliminate drift caused by errors; post-anomaly repositioning is generally applied to pose confirmation after positioning loss. Therefore, under the two scenes, the map semantic information acquired in the positioning process before the pose is determined again is relatively accurate, and can be directly used as a target semantic map. Semantic objects in the object semantic map are referred to as standard semantic objects in this application.

After the target semantic map of the target equipment and the semantic map to be matched are determined, the features (semantic targets) in the two maps can be matched, and the pose corresponding to the semantic map to be matched is determined.

The following steps are a way of semantic object matching provided by the present application.

S202, based on the semantic map to be matched, determining a minimum spanning tree based on the undirected complete graph, wherein a node of the minimum spanning tree is a semantic target to be matched.

By constructing the minimum spanning tree corresponding to the semantic map to be matched, the feature information of each semantic target to be matched in the semantic map to be matched can be converted into the structure information of the minimum spanning tree.

S203, constructing a search tree based on the target semantic map, wherein the nodes of the search tree are standard semantic targets.

And taking each standard semantic target in the target semantic map as a node to construct a search tree.

And S204, determining a growth tree matched with the minimum spanning tree from the constructed search tree.

In this application, matching refers to that the feature information of each semantic object is basically consistent (basically, the same object can be determined to be different embodiments in two semantic maps). Then, the constructed spanning tree is matched with the minimum spanning tree, that is, the feature information of the standard semantic target corresponding to each node of the spanning tree is basically consistent with the feature information of the semantic target to be matched corresponding to each node of the minimum spanning tree.

Specifically, traversal can be started from the root node of the minimum spanning tree, and the standard semantic objects matched with the minimum spanning tree are determined one by one from the search tree to serve as the nodes of the spanning tree, so that the spanning tree is constructed.

Furthermore, the process of matching semantic objects in the two maps can be converted into a process of growing trees.

In some embodiments, the lookup tree may be a K-dimensional tree. The structure of the K-dimensional tree and the related construction algorithm can optimize the structure of the search tree, and when the growth tree is determined, the operation times are less, the operation pressure is less, and the search efficiency of each node is further improved.

S205, determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree.

According to the corresponding relation between each node in the built spanning tree and each node in the minimum spanning tree, the matching relation between the standard semantic target and the semantic target to be matched can be determined, namely, which standard semantic target in the standard semantic map is matched with which semantic target to be matched in the semantic map to be matched can be determined.

Thus, the semantic target matching process between the target semantic map and the semantic map to be matched is completed.

After the matching relationship between the standard semantic target and the semantic target to be matched is determined, the pose change of each semantic target to be matched relative to the standard semantic target matched with the semantic target can be determined. And determining the pose of the semantic target to be matched in the semantic target map to be matched, namely the current pose of the target equipment, by combining the poses of the standard semantic target in the target semantic map.

The global relocation method provided by the embodiment comprises the following steps: determining a target semantic map and a semantic map to be matched of target equipment, wherein the target semantic map comprises a plurality of standard semantic targets, and the semantic map to be matched comprises a plurality of semantic targets to be matched; determining a minimum spanning tree based on an undirected complete graph based on a semantic map to be matched, wherein a node of the minimum spanning tree is a semantic target to be matched; constructing a search tree based on the target semantic map, wherein nodes of the search tree are standard semantic targets; determining a growth tree matched with the minimum spanning tree from the constructed search tree; and determining the pose of the target equipment according to the matching relation between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree. The problem of target matching is converted into the problem of growth tree growth, and the growth of the growth tree can select the optimal node based on all standard semantic targets in the target semantic map and grow into the growth tree which is optimally matched with the minimum generation tree. The target matching can be carried out based on all semantic targets in the target semantic map, so that the global relocation is really realized, and the global positioning efficiency of the intelligent equipment is improved.

In some specific implementations, the determining, based on the semantic map to be matched, the minimum spanning tree based on the full map may include: aiming at each semantic target to be matched, taking the semantic target to be matched as a node, and constructing an undirected complete graph; aiming at any two semantic objects to be matched, determining the weight of an edge between nodes constructed by the two semantic objects to be matched according to translation information of the two semantic objects to be matched; and determining the minimum spanning tree of the undirected complete graph based on the weight of each edge in the undirected complete graph.

Namely, each semantic target to be matched in the map to be matched is taken as a node of the undirected complete map, and translation information of two semantic targets to be matched is taken as the weight of an edge corresponding to a node corresponding to the two semantic targets to be matched.

Translation information between the two semantic objects to be matched can be acquired from the semantic map to be matched. For example, according to the relative positions of the two semantic objects to be matched in the semantic map to be matched, translation vectors corresponding to the two semantic objects to be matched can be determined as translation information.

In some implementations, the determining the minimum spanning tree of the undirected full graph based on the weights of the edges in the undirected full graph may include: enumerating all spanning trees corresponding to the undirected complete graph; and calculating the sum of the weights corresponding to all edges in each spanning tree, and determining the spanning tree with the minimum sum of the weights as the minimum spanning tree. Just to name one of the methods for generating the minimum spanning tree, in other implementations, the minimum spanning tree may also be determined using a Prim Algorithm (Prim Algorithm), a Kruskal Algorithm (Kruskal Algorithm), or a variation of other spanning tree algorithms. A detailed description thereof will be omitted.

In one specific example, based on the set to be matched

The pure spatial structure information of (the set of semantic objects to be matched) constructs an undirected complete graph. The calculation of the weights of the edges in the complete map may utilize only translation information

The corresponding weighting function can be referred to the following formula (1):

(1)；

wherein the content of the first and second substances,

the representation is composed of a pair of semantic objects

And

weights of edges between the constructed nodes;

、

separately representing semantic objects

And

the translation vector of (2).

Finally generate one

Minimum weighted tree of

The tree is in the set to be matched

Of an arbitrary complete tree

The medium weight is the smallest, and the mathematical expression can refer to formula (2):

(2)；

by constructing the minimum spanning tree, the measurement error introduced by the observation space scale range in the process of converting the semantic map into the graph can be reduced as much as possible. And the disorder caused by random matching can be avoided by matching based on the minimum spanning tree, the rigidity requirement caused by graph matching is avoided, the space-time characteristic of observation addition is met, the error accumulation is met, and the error propagation relation can be well met.

In some embodiments, the above-mentioned manner of determining the grown tree matching the minimum spanning tree from the constructed search tree may include: constructing a survival rule tree based on a survival rule corresponding to each branch in the minimum spanning tree; and determining a growing tree from the constructed search tree based on the survival rule tree, wherein the growing tree meets the survival rule represented by the survival rule tree.

The feature information of each semantic target to be matched in the semantic map to be matched represented by the minimum spanning tree can be used as a constraint condition (survival rule) to construct a survival rule tree. Based on the survival rule represented by the survival rule tree, the construction of the growth tree can be guided, and finally the growth tree completely conforming to the survival rule tree is constructed. And if the target semantic map and the semantic map to be matched have no ambiguity, finally constructing a growth tree completely conforming to the survival rule tree from the search tree.

In some embodiments, the above-mentioned survival rule for each branch comprises: the survival rule corresponding to each branch comprises the following steps: the relative pose and the pre-estimated error variance of the node pair corresponding to each branch; correspondingly, the above-mentioned survival rule based on each branch in the minimum spanning tree is constructed, and includes: aiming at the node pair corresponding to each branch in the minimum spanning tree, calculating the relative pose corresponding to the node pair; aiming at the node pair corresponding to each branch in the minimum spanning tree, calculating the pre-estimated error variance corresponding to the node pair; generating a survival rule corresponding to each branch by using the position of each branch in the minimum spanning tree, the relative pose of the node pair corresponding to the branch and the estimated error variance; and determining a survival rule tree according to the survival rule corresponding to each branch in the minimum spanning tree.

And corresponding the survival rule corresponding to each branch in the minimum spanning tree to the branch to generate the survival rule tree corresponding to the minimum spanning tree. The survival rule corresponding to each branch actually comprises the characteristic information of two nodes corresponding to the branch. In some embodiments, the feature information of two nodes can be represented by the relative pose between the two nodes and the variance of the estimated error between the two nodes. The relative pose between the two nodes can be calculated and determined through the relative positions of the two nodes; the estimation error variance between two nodes refers to the variance relationship of the estimation positioning errors of the semantic targets corresponding to the two nodes, and can be estimated according to the errors of the positioning system of the target equipment.

In some scenarios, the semantic map to be matched and the target semantic map have attribute features of each semantic target simultaneously. In this scenario, the survival rule corresponding to each branch may further include: attribute characteristics of each node. Correspondingly, constructing a survival rule tree based on the survival rule corresponding to each branch in the minimum spanning tree may further include: determining attribute characteristics of each node corresponding to each branch in the minimum spanning tree; and generating a survival rule corresponding to the branch by using the position of each branch in the minimum spanning tree, the relative pose of the node pair corresponding to the branch, the estimated error variance and the attribute characteristics of each node.

The attribute features of the semantic object may include, but are not limited to, size, shape, category, color, etc. of the object. The corresponding specific attribute may be referred to as an attribute feature value, for example, the color is yellow, the shape is a cube, and the like.

It can be understood that the more feature information is applied, the more detailed the constraints of the survival rule are, and when the growth tree is determined from the constructed search tree based on the survival rule tree, the processing efficiency is higher, and the accuracy is also higher.

Specifically, the calculating the variance of the estimated error corresponding to the node pair for the node pair corresponding to each branch in the minimum spanning tree may include: determining an observation error and a distance error corresponding to target equipment; aiming at the node pair corresponding to each branch in the minimum spanning tree, determining the distance between the targets to be matched corresponding to the node pair; and determining the pre-estimated error variance corresponding to the node pair according to the observation error, the distance error and the distance corresponding to the node pair.

Because the sensor and the processing algorithm of the target device have certain errors, the position information of each semantic target in the map also has errors when the target device generates the semantic map by using the information acquired by the sensor. This error is referred to herein as an observation error. In addition, as the travel distance increases, the target device has accumulated errors, which also causes errors in the positioning result. This error is referred to herein as a range error. By analyzing the historical positioning data of the target device, observation errors and distance errors can be evaluated. The estimated observation error and distance error can be used for well estimating the positioning error of each semantic target in the semantic map, and the estimation error is the estimation error. In this embodiment, a variance between the estimation errors of the two semantic objects (estimation error variance) is used as one of the growth rules of the branches corresponding to the two semantic objects.

In one implementation, the calculation of the variance of the prediction error may be performed using equation (3). The estimated error in this implementation can be used to calculate the translational error and the rotational error. The formula describes two matching targets

The method for calculating the variance of the estimated error between the two targets represents the variance level between the two target errors.

(3)；

Wherein the content of the first and second substances,

for two matching targets

The estimated error variance between;

for two matching targets

In m;

uncertainty for the system itself as a function of distance traveled (distance error);

the inherent observation error (observation error) can be obtained by evaluating the sensor precision and the calculation error of the system.

The range error may be derived from the error propagation characteristics of the target device itself, and in one implementation, is composed of rotation and translation

. The observation error can be obtained according to the accuracy of the sensor of the target device and the calculation error, and in one implementation mode, the observation error is

。

The scheme of the embodiment can adopt a method of respectively calculating and respectively comparing errors on the translation amount and the rotation amount. In other implementations, methods incorporating computational comparisons may also be employed.

Corresponding to the scenario with attribute features described later, the construction process of the rule-of-survival tree can refer to the following.

Traversing each pair of nodes in a minimum spanning tree

Calculating

And

the calculation method can refer to formula (4):

(4)；

traversing each pair of nodes of the minimum spanning tree

Calculating the theoretical level of the estimated error variance of each node except the root node and the previous node by adopting the formula (3)

。

Recording attribute characteristics corresponding to each node

。

The growth rule corresponding to each pair of nodes

，

，

，

Corresponding to the branch of the minimum spanning tree to form a survival rule of the branch

. Rebinding index number

Forming a growth rule of each branch, thereby constructing a rule tree

. In some embodiments, a slack variable may also be set for each rule of survival

The range limited by the survival rule is properly enlarged to avoid that the correct matching result is judged not to conform to the survival rule because of overlarge error.

According to a minimum spanning tree, fromSurvival rule of starting tree root and constructing branch by branch

And finally forming a survival rule tree. So every branch formed by adding a pair of nodes must satisfy that the branches already existing in the rule tree must contain one node of the newly constructed branch. The complete survival rule tree can be aligned and constructed according to equation (5):

(5)；

wherein the content of the first and second substances,

representing a survival rule tree;

the representation tree includes a set of node numbers

Forming branches; symbol'

' means that the left end is updated with its right end.

The construction of the rule-of-survival tree can be completed.

In some embodiments, the above-mentioned manner for determining a growing tree from the constructed search tree based on the survival rule tree, where the growing tree satisfies the survival rule represented by the survival rule tree, includes: selecting a node from the minimum spanning tree as a tree root node; selecting at least one standard semantic target with the same attribute characteristics as the root nodes of the tree from the target semantic map as a seed of the growth tree; traversing each node of the minimum spanning tree upwards from the root node of the tree; for each node of the minimum spanning tree, performing the following steps: determining branches to be grown corresponding to the nodes and branch nodes forming the branches to be grown with the nodes; and searching a standard semantic target matched with the branch nodes from the search tree as the growth nodes of the growth tree according to the survival rule tree. And after traversing the minimum spanning tree, determining a growing tree according to the seeds and the corresponding growing nodes.

In some implementations, a node may be randomly selected from the minimum spanning tree as a tree root node. After the tree root node is determined, according to the attribute characteristics of the tree root node, the node matched with the tree root node can be determined from the search tree and used as the seed of the growth tree. The seed may be one or more. A growth tree may be generated based on each seed.

And selecting seeds from the target semantic map. The specific way can be that the target semantic map is traversed

Finding out the nodes of all elements in the tree root

Semantic objects of which the categories belong to are completely consistent

Flower bodies are used as seeds needed by forests

Representing a set of seeds, which can be represented by equation (6):

(6)；

wherein the content of the first and second substances,

to represent

The corresponding seed.

The implementation flow of this embodiment may refer to fig. 3. Each node of the minimum spanning tree is traversed starting from the tree root node. When traversing to each node of the minimum spanning tree, determining the branch to be grown corresponding to the node according to the growing direction, and calling another node on the branch to be grown as a branch node. The branch node corresponds to the next growing node to grow on the growing tree. And finding the standard semantic target matched with the branch node from the search tree corresponding to the target semantic map as the next growth node of the growth tree. And if no branch consistent with the growth direction exists on the node, continuously traversing the next node. If the tree branch to be grown corresponding to the node has a plurality of branches, the growing nodes corresponding to the tree branch nodes are searched one by one. After traversing the minimum spanning tree, each growing node on the growing tree based on the seed growth is determined, and the growing of the growing tree is completed.

For ease of understanding, further description is provided with reference to fig. 4. In fig. 4, a is a minimum spanning tree and b is a growth tree. There are three sections a1, a2, a3 in a, where a1 is determined to be a tree root node. b1 in b is one of the seeds found according to a 1. The predetermined direction of traversal is shown by the dashed arrow in a. The growth process of the growing tree b is as follows: at the beginning, based on the two branch nodes a2 and a3 corresponding to the node a1, the b2 and b3 which are matched with the search tree are found out from the search tree and are used as two new growth nodes in the b. Traversing down to a2, a2 has no corresponding branch to grow, moving to the next node a3, a3 being the last node of the minimum spanning tree, and ending. And finishing the traversal process. b, finishing the growth.

In other implementations, in order to improve the generation efficiency of the search tree, a special node in the minimum spanning tree may also be selected as the root node. In one implementation manner, the selecting a node from the minimum spanning tree as a tree root node includes: for each node in the minimum spanning tree, determining the attribute characteristics of the node; determining a first number corresponding to nodes with the same attribute characteristic value as the node in the minimum spanning tree aiming at each attribute characteristic of the node; determining a second number corresponding to a standard semantic target with the same attribute characteristic value as the node in the target semantic map aiming at each attribute characteristic of the node; determining the specificity score of the node according to the first quantity and the second quantity corresponding to each attribute feature of the node; and selecting the node with the highest specificity score as a tree root node. Namely, a tree root node is determined from the nodes of the minimum spanning tree according to the distribution condition of the attribute characteristic value of each semantic target in the target semantic map and the semantic map to be matched.

Wherein the specificity score can be a value negatively correlated to the first quantity and the second quantity. For example, the specificity score is negatively correlated with the sum of all of the first quantity and all of the second quantity. Thus, when the repeatability of each attribute feature value of a certain node in the minimum target semantic map and the semantic map to be matched is low, the higher the specificity score is, the more special the node is. Because the number of semantic objects with the same attribute characteristic value is less, the difficulty in searching the matched semantic objects is lower, the matching efficiency is higher, and the growth process is faster.

In a specific implementation, the first number and the second number may be embodied by normalized histograms. Specifically, for any attribute feature of any semantic target to be matched, determining quantity information corresponding to the attribute feature in a first normalized histogram and quantity information corresponding to the attribute feature in a second normalized histogram; and aiming at any semantic target to be matched, determining the specificity score of the semantic target to be matched based on the quantity information corresponding to each attribute feature of the semantic target to be matched in the first normalized histogram and the quantity information corresponding to each attribute feature in the second normalized histogram.

The method for determining the specificity score of the semantic target to be matched according to the attribute characteristics of the semantic target to be matched, the first normalized histogram and the second normalized histogram includes the following steps: determining quantity information corresponding to the attribute features in the first normalized histogram and quantity information corresponding to the attribute features in the second normalized histogram aiming at any attribute feature of any semantic target to be matched; and aiming at any semantic target to be matched, determining the specificity score of the semantic target to be matched based on the quantity information corresponding to each attribute feature of the semantic target to be matched in the first normalized histogram and the quantity information corresponding to each attribute feature in the second normalized histogram.

In some implementations, the tree root selection process is as follows.

Calculating a normalized histogram (histogram) of class-number (category-count) in the target semantic map, using

It is shown that,

representing abstract properties

In

The corresponding histogram value. In the case of not replacing the target map, the step (calculation process of the target semantic map correlation histogram) only needs to calculate one round, and the calculation method can refer to formulas (7), (8) and (9):

(7)；

(8)；

(9)；

wherein the content of the first and second substances,

the operator being an object

In a category

(ii) attribution of (a); '=' is same type matching judgment, the type is the same 1, and the type is different, the type is 0;

is shown in

Middle affiliation

The value of (a) is,

is shown in

In the histogram, the highest attributed value, i.e. vector

Maximum value of (d);

is represented by a subscript

Traversed

The object of (1);

representing attribute features without loss of generality

Is composed of

One of the abstract properties of (1) is,

is composed of

Corresponding to a different distinction of certain abstract properties (e.g., defining colors as

The color attributes are respectively corresponding to red, green and blue

）。

A normalized histogram (histogram) of the category-number (category-count) in the map to be matched is calculated.

And

the calculation method of (2) is the same as that of the formulas (8) and (9),

the calculation of (2) only needs to change the source of the object, as in equation (10):

(10)。

defining a specificity score and using the English letters of the flower body

And (4) showing. Calculating the scores of all the objects in the set to be matched, as shown in formula (11):

(11)。

wherein the content of the first and second substances,

is composed of

Middle target

A specificity score for all categories to which it belongs;

is a natural constant;

is m of

Attribute is

The normalized histogram of (a);

is m of

The attribute is in

Normalized histogram.

Selecting the semantic object with the highest specificity score

By using

To represent

Root (root) as a minimum spanning tree, then

Satisfy the publicFormula (12):

(12)；

wherein

Represents

The specificity score vectors for all subjects in (1).

In this embodiment, the calculation of the specificity score comprehensively considers the object to be matched and all semantic objects in the target map. Therefore, the selected tree root is the semantic target in the semantic map to be matched, which is most likely to be matched with the semantic target in the target map. Of course, in other implementations, a weighted sum, or other statistical idea, may also be considered for use in the calculation of the present score.

In some embodiments, the above-mentioned manner for finding the standard semantic object matching with the branch node from the search tree as the growth node of the growth tree according to the rule-of-survival tree may include: calculating an external cube of the minimum spanning tree; calculating the distance from the root node of the tree to each vertex of the cube; taking the maximum distance from the tree root node to each vertex of the cube as a search radius; for each seed, determining a corresponding sub search tree from the search trees based on the search radius; and aiming at each node of the minimum spanning tree, searching a standard semantic target matched with the node from each sub-search tree as a growth node of the growth tree according to a survival rule corresponding to the node.

The sub search trees of the search tree are constructed in a certain range corresponding to each seed, so that the search range can be reduced to a certain extent, and the search efficiency is improved. Wherein this range (search radius) may be determined from the range of the minimum spanning tree. As long as the range of the sub search tree is not less than the range of the minimum spanning tree, the nodes in the sub search tree can be ensured to be enough for the growth of the growth tree.

One way to determine the range of the sub-tree is to determine the range of the minimum spanning tree as the search radius according to the circumscribed cube of the minimum spanning tree.

Circumscribed cube for calculating minimum spanning tree (Oriented bounding box, OBB)

The calculation using regular bounding boxes instead of the irregular minimum spanning tree itself is more efficient and simpler.

Calculating the distance from the tree root to the vertex (8 vertexes) of the cube, and taking the maximum distance as the search radius

Reference may be made to equation (13):

(13)；

wherein the content of the first and second substances,

to represent

The point represented;

each vertex of a circumscribed cube representing a minimum spanning tree;

is the root of a tree.

Constructing K dimensional tree (KDtree) of target semantic map, and combining

As the search radius, a sub KDtree (lookup tree) satisfying the size of the search radius range is constructed for each seed, and

and (4) showing.

Wherein, lower case letters

The number of sub-KDtrees is indicated; and is

Wherein

As already explained above, it is emphasized here that this is a target semantic map.

It can be seen from this that there may be a plurality of seeds consistent with the attribute characteristics of the root node, and as many seeds can generate as many growing trees, and as many growing trees form a forest, the growing of the growing trees in this application can also be regarded as forest growing.

In the above embodiment, according to the survival rule tree, searching the standard semantic object matched with the branch node from the search tree as the growth node of the growth tree may specifically include: aiming at the sub search tree corresponding to each seed, the following steps are executed: determining a survival rule of branches to be grown from the survival rule tree; determining a current node in a sub-search tree; searching nodes positioned in the searching radius of the current node of the searching tree in the sub searching tree to serve as a first candidate node set; determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown; taking the final candidate node as a growth node of the growth tree; and if the final candidate node is not determined, deleting the growth tree corresponding to the seed.

The description is made corresponding to the scenario of fig. 4 described above. For example, currently, the growth node corresponding to a2 is searched, and the determination process of the growth node corresponding to the seed b1 is taken as an example. First, the survival rule of the branch a1a2 is determined from the survival rule tree. Because the a1 node in a is currently traversed and the branch a1a2 is to be grown, it can be determined that the current node in the sub-lookup tree is the most recently grown node b1 in the grown tree b. And searching the nodes positioned in the search radius of b1 from the sub search tree to serve as a first candidate node set. From the set, the node that conforms to the survival rule of branch a1a2 is determined as the final candidate node, i.e., the latest growing node b 2.

Of course, since seed b1 is not necessarily the matching target of a1, it is likely that no node that meets the survival rule of branch a1a2 can be found in the first set of candidate nodes, thus indicating that b is not a growing tree that matches a, and that this tree can be deleted. And in the subsequent continuous growth process, the corresponding growth treatment is not carried out on the b, so that the calculated amount is reduced.

Wherein, from the rule of survival tree, the specific manner of determining the rule of survival of the branch to be grown may include: and determining the relative pose of the node pair corresponding to the branch to be grown and the prediction error variance from the survival rule tree. Correspondingly, determining a final candidate node from the first candidate node set according to the survival rule of the branch to be grown, including: determining the relative pose of each node in the first candidate node set and the current node of the sub search tree; determining whether the node is matched with a branch node or not according to the relative pose of the current node of the node and the sub search tree, the relative pose of the node pair corresponding to the branch to be grown and the estimated error variance; and if the node is matched with the branch node, determining the node as a final candidate node.

The branch formed by the candidate node and the current node in the spanning tree is referred to as a candidate branch. The scheme of the embodiment is to determine whether the branch to be grown is matched with each candidate branch by comparing the pose relationship represented by the branch to be grown in the minimum spanning tree with the pose relationship of each candidate branch in each spanning tree, and further determine whether the branch node is matched with each candidate node.

Specifically, whether to match can be determined as follows. Determining a mismatch error according to the relative poses of the node and the current node of the sub search tree; determining an actual error variance according to the mismatch error; and determining whether the node is matched with the branch node or not according to the actual error variance and the estimated error variance and by combining chi-square test.

According to the actual error variance between two targets in the target semantic map, chi-square test is carried out on the actual error variance and the estimated error variance of the target equipment, and whether the difference is within a preset range is verified. If the difference is within the preset range, it can be determined that two targets in the target semantic map are matched with two targets in the semantic map to be matched. The principle on which the scheme is based is that if two targets in the target semantic map are matched with two targets in the semantic map to be matched, the error variance of the two targets between the two semantic maps can follow the rule of the estimated error variance.

In one particular implementation, assuming that the targets are all represented in a fixed coordinate system in the respective sets, the actual error variance between the targets is calculated as equation (14):

(14)；

wherein, the first and the second end of the pipe are connected with each other,

representing a certain amount in relative pose, which may be a displacement amount

Or the posture quantity

；

Presentation map

Medium semantic object

A quantitative representation of a relative pose relationship therebetween;

representation collection

Medium semantic object

An inverse of a relative pose relationship between them, can also be used

The representation, the two are unambiguous,

is that

To pair

In (1)

A speculative association of objects; binary operation'

' denotes the addition operation defined in the algebraic system in which the variable is located;

is represented by a set

Medium semantic object

And

medium semantic object

Is a mismatch error of

And

a measure of difference between the two;

is the actual error variance calculated from the mismatch error.

The chi-square test is a single-tailed test, and the significance level refers to the risk level to be borne by the H0 hypothesis, namely the probability level, if the H0 hypothesis is true. The significance level is typically chosen to be 0.05 or 0.01, meaning that there is a 95% or 99% probability of being H1 in the reject domain, so in this case being the reject domain then excluding H0 will result in the chi-square being generally biased towards the H0 hypothesis.

In contrast, in the practice of the present invention, there is no bias toward H0 but rather toward H1. So that accepting the H0 hypothesis with H0 hypothesis as true entails a very low risk level, the chi-square test of the present scheme takes the mean of the errors as the hypothesis, takes 'two-correlation' as H0, and selects the left-tailed test, i.e. the significance level

。

In this embodiment, the semantic object is a planar semantic object, and the pose of the semantic object is expressed separately, and the translation amount is calculated

Is 2 degrees of freedom, the amount of posture

Is 1 degree of freedom. The rejection areas under different degrees of freedom can be obtained through checking a chi-square distribution table. At the significance level of 0.9,

and

the test of (a) corresponds to cut-off values of 0.211 and 0.016, respectively, as follows:

(15)。

by coefficient of relaxation variables

，

To set the linear buffer for hypothesis testing against errors of unknown origin, the embodiment uses

As a relaxation variable coefficient.

In between

The (chi-square) magnitude can be approximated by the calculation of equation (16), which represents the difference between the actual error variance and the estimated error variance (rather than estimating the same distribution):

) (16)。

the present embodiment is directed to a determination method in a scene without attribute feature information, and the determination method in a scene with attribute feature information may refer to the next embodiment.

Under the scene with attribute characteristic information, the relative pose of the node pair corresponding to the branch to be grown, the estimation error variance and the attribute characteristics of the branch node can be determined from the survival rule tree. Correspondingly, the determining the final candidate node from the first candidate node set according to the survival rule of the branch to be grown includes: determining whether the attribute characteristics of the nodes are the same as the attribute characteristics of the branch nodes or not aiming at each node in the first candidate node set; if the attribute characteristics of the nodes are different from those of the branch nodes, deleting the nodes from the first candidate node set to obtain a second candidate node set; determining the relative pose of the node and the current node of the sub search tree aiming at each node in the second candidate node set; determining whether the node is matched with a branch node or not according to the relative pose of the current node of the node and the sub search tree, the relative pose of the node pair corresponding to the branch to be grown and the estimated error variance; and if the node is matched with the branch node, determining the node as a final candidate node.

In the scenario of this embodiment, due to the addition of the attribute features, the first candidate node set may be preliminarily screened, so that the processing procedure may be greatly simplified, and the processing efficiency may be improved.

In one specific embodiment, the forest growth process is as follows.

Traversal survival rule tree

And each with seeds

Formed child lookup tree

Growth starts from the seed. The goal of each seed is to gradually branch and leave, grow into a tree that satisfies the rule of survival, and then

Representing the growing tree grown from the seed.

The growth steps are as follows:

beginning: starting from the first node in the survival Law Tree (root node), each cycle is used

Representing a current node; find out the node (branch node) corresponding to it to form the branch

Is shown, i.e.

And acquire all the rules thereof

{

，

，

，

}。

First-level candidate node set: by using

Represented as the current node in the long tree, from

Find all the current node's satisfaction to

Being a node of radius, using

It is shown that,

. Wherein

Is shown by

A corresponding set of candidate nodes.

Second level candidate node set: for sets

All nodes in (1), by category

Exclusions are made using an updated set of nodes that satisfy equation (17)

The set is then evolved into a second level candidate node set.

(17)。

And (3) final candidate nodes: for collections

All remaining nodes in (1) are calculated using equation (15)

And retaining the result, and according to the critical value obtained by the table lookup, the node which meets the formula (18) becomes the final candidate node.

(18)。

To this end, for a real physical space and a perfect map, the final candidate point is often only one, if extreme conditions occur, i.e. multiple passes chi-square tests (theoretically not present unless the map is ambiguous), thenAccording to

Sorting and selecting values

Minimum size

As final candidate nodes.

Killing all trees that failed the rule of survival test

And updating the forest, as in equation (19):

(19)。

the growth steps are cycled, and the n trees which survive last

Are the intersection of the map to be matched and the target map. Finally, unfolded

Each tree can be represented as having

Matching solution set of individual elements

。

If it is not

Is provided with

And then there is no solution. If it is not

Is provided with

There are multiple solution sets, but the correct solution must be. Separate solutions may be selected.

For the obtained matching solution set

Nodes according to the rule of survival

Restoring the corresponding relation to obtain the one-to-one corresponding relation between the map to be matched and the semantic target in the target map

。

Subsequently, the pose relationship between the maps determined by the tree can be solved by constructing an optimization problem

The solution can be solved using equation (20):

(20)。

other algorithms such as ICP may also be utilized for the solution method.

For a map with a large range of deformation due to accumulated error of temporary map estimation, only the n latest corresponding matches in the tree can be taken

As a nearest real semantic observation target, discarding nodes with too long time or too long distance, still constructing a problem according to a formula (20) and solving the problem, namely obtaining a pose almost irrelevant to recursion errors.

The theoretical semantic observation target requirement is at least 1, and the number of required observations at corresponding precision is related to the error level of the system. In the case of 6dof, if the pose information needs to be discarded to obtain the effect of accelerating the solution, at least 3 non-collinear targets are needed, and in the case of discarding the pose information of the semantic target, the problem becomes formula (21):

(21)。

fig. 5 is a schematic structural diagram of a global relocation apparatus according to an embodiment of the present application, and as shown in fig. 5, a global relocation apparatus 500 according to this embodiment includes: semantic object determination module 501, minimum spanning tree determination module 502, search tree determination module 503, growth tree determination module 504, pose determination module 505.

The semantic target determining module 501 is configured to determine a target semantic map and a to-be-matched semantic map of a target device, where the target semantic map includes a plurality of standard semantic targets and the to-be-matched semantic map includes a plurality of to-be-matched semantic targets;

a minimum spanning tree determining module 502, configured to determine a minimum spanning tree based on an undirected complete graph based on a to-be-matched semantic map, where a node of the minimum spanning tree is a to-be-matched semantic target;

a search tree determining module 503, configured to construct a search tree based on the target semantic map, where nodes of the search tree are standard semantic targets;

a growing tree determining module 504, configured to determine, from the constructed search tree, a growing tree that matches the minimum growing tree;

and a pose determining module 505, configured to determine a pose of the target device according to a matching relationship between the standard semantic target corresponding to the spanning tree and the semantic target to be matched corresponding to the minimum spanning tree.

Optionally, the minimum spanning tree determining module 502 is specifically configured to:

aiming at any two semantic targets to be matched, determining the weight of an edge between nodes constructed by the two semantic targets to be matched according to translation information of the two semantic targets to be matched;

the minimum spanning tree of the undirected full graph is determined based on the weights of the edges in the undirected full graph.

Optionally, the growing tree determining module 504 is specifically configured to:

constructing a survival rule tree based on a survival rule corresponding to each branch in the minimum spanning tree;

and determining a growing tree from the constructed search tree based on the survival rule tree, wherein the growing tree meets the survival rule represented by the survival rule tree.

the spanning tree determining module 504 is specifically configured to, when constructing the rule-of-survival tree based on the rule-of-survival corresponding to each branch in the minimum spanning tree:

aiming at the node pair corresponding to each branch in the minimum spanning tree, calculating the pre-estimated error variance corresponding to the node pair;

Optionally, the growing tree determining module 504 is specifically configured to, when calculating the prediction error variance corresponding to the node pair for the node pair corresponding to each branch in the minimum spanning tree:

determining an observation error and a distance error corresponding to target equipment;

Optionally, the growing tree determining module 504 is configured to determine the growing tree from the constructed search tree based on the survival rule tree, and when the growing tree satisfies the survival rule represented by the survival rule tree, specifically:

selecting a node from the minimum spanning tree as a tree root node;

selecting at least one standard semantic target which has the same attribute characteristics with the root nodes of the tree from the target semantic map as a seed of the growth tree;

traversing each node of the minimum spanning tree upwards from the root node of the tree;

for each node of the minimum spanning tree, performing the following steps:

searching a standard semantic target matched with the branch nodes from the search tree as growth nodes of the growth tree according to the survival rule tree;

Optionally, when the growing tree determining module 504 selects one node from the minimum spanning tree as the tree root node, it is specifically configured to:

determining attribute characteristics of the nodes aiming at each node in the minimum spanning tree;

determining a first number of nodes corresponding to the nodes with the same attribute feature value as the node in the minimum spanning tree aiming at each attribute feature of the node;

determining a second number corresponding to a standard semantic target with the same attribute feature value as the node in the target semantic map aiming at each attribute feature of the node;

and selecting the node with the highest specificity score as a tree root node.

Optionally, the growing tree determining module 504 is specifically configured to, when searching for a standard semantic object matched with the branch node from the search tree as a growing node of the growing tree according to the rule-of-survival tree:

calculating an external cube of the minimum spanning tree;

calculating the distance from the root node of the tree to each vertex of the cube;

and aiming at each node of the minimum spanning tree, searching a standard semantic object matched with the node from each sub-search tree as a growth node of the growth tree according to a survival rule corresponding to the node.

determining a survival rule of branches to be grown from the survival rule tree;

determining a current node in a sub-search tree;

searching nodes positioned in the searching radius of the current node of the searching tree in the sub searching tree to serve as a first candidate node set;

taking the final candidate node as a growth node of the growth tree;

Optionally, when determining the survival rule of the branch to be grown from the rule of survival tree, the growing tree determining module 504 is specifically configured to:

when determining the final candidate node from the first candidate node set according to the survival rule of the branch to be grown, the growing tree determining module 504 is specifically configured to:

determining the relative pose of each node in the first candidate node set and the current node of the sub search tree;

determining whether the node is matched with a branch node or not according to the relative pose of the current node of the node and the sub search tree, the relative pose of the node pair corresponding to the branch to be grown and the estimated error variance;

determining the relative pose of the node pair corresponding to the branch to be grown, the estimation error variance and the attribute characteristics of the branch node from a survival rule tree;

determining whether the attribute characteristics of the nodes are the same as the attribute characteristics of the branch nodes or not aiming at each node in the first candidate node set;

if the attribute characteristics of the nodes are different from those of the branch nodes, deleting the nodes from the first candidate node set to obtain a second candidate node set;

determining the relative pose of the node and the current node of the sub search tree aiming at each node in the second candidate node set;

Optionally, the growing tree determining module 504 is specifically configured to, when determining whether a node matches a branch node according to the relative pose of the node and the current node of the sub search tree, the relative pose of the node pair corresponding to the branch to be grown, and the estimated error variance:

determining an actual error variance according to the mismatch error;

The apparatus of this embodiment may be configured to perform the method of any of the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 6, an electronic device 600 according to the embodiment may include: a memory 601 and a processor 602.

The memory 601 has stored thereon a computer program that can be loaded by the processor 602 and executed to perform the method in the above-described embodiments.

The processor 602 is coupled to the memory 601, such as via a bus.

Optionally, the electronic device 600 may also include a transceiver. It should be noted that the transceiver in practical application is not limited to one, and the structure of the electronic device 600 is not limited to the embodiment of the present application.

The Processor 602 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 602 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

A bus may include a path that transfers information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The Memory 601 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

The memory 601 is used for storing application program codes for executing the scheme of the application, and the processor 602 controls the execution. The processor 602 is configured to execute application program code stored in the memory 601 to implement the content shown in the foregoing method embodiments.

Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. But also a server, etc. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present application.

The electronic device of this embodiment may be configured to perform the method of any of the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

The present application also provides a computer readable storage medium storing a computer program that can be loaded by a processor and executed to perform the method as in the above embodiments.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Claims

1. A global relocation method, comprising:

2. The method according to claim 1, wherein the determining a minimum spanning tree based on a full graph based on the semantic map to be matched, a node of the minimum spanning tree being the semantic target to be matched, comprises:

3. The method of claim 1 or 2, wherein the determining the growth tree matching the minimum spanning tree from the constructed search tree comprises:

4. A method as claimed in claim 3, wherein the survival rules for each branch include: the relative pose and the pre-estimated error variance of the node pair corresponding to each branch;

5. The method of claim 4, wherein the corresponding law of survival for each branch further comprises: attribute characteristics of each node;

6. The method according to claim 4 or 5, wherein the calculating, for the node pair corresponding to each branch in the minimum spanning tree, the estimated error variance corresponding to the node pair comprises:

7. The method as claimed in claim 3, wherein said determining a growth tree from said constructed said search tree based on said survival rules tree, said growth tree satisfying the survival rules represented by said survival rules tree, comprises:

selecting a node from the minimum spanning tree as a tree root node;

for each node of the minimum spanning tree, performing the following steps:

8. The method of claim 7, wherein the selecting a node from the minimum spanning tree as a tree root node comprises:

determining, for each attribute feature of the node, a first number of nodes in the minimum spanning tree corresponding to the same value of the attribute feature of the node;

and selecting the node with the highest specificity score as a tree root node.

9. The method according to claim 7 or 8, wherein said searching for a standard semantic object matching said branch node from said search tree as a growth node of said growth tree according to said survival rules tree comprises:

calculating a circumscribed cube of the minimum spanning tree;

calculating the distance from the tree root node to each vertex of the cube;

10. The method of claim 9, wherein said searching for a standard semantic object matching said branch node from said search tree as a growth node of said growth tree according to said rule-of-survival tree comprises:

aiming at the sub-search tree corresponding to each seed, the following steps are executed:

determining a current node in the sub-lookup tree;

taking the final candidate node as a growth node of the growth tree;

11. The method as claimed in claim 10, wherein said determining the survival law of said branch to grow from said tree of survival laws comprises:

12. The method as claimed in claim 10, wherein said determining a survival rule of said branch to be grown from said tree of survival rules comprises:

13. The method of claim 11 or 12, wherein the determining whether the node matches the branch node according to the relative pose of the node and the current node of the sub-tree, the relative pose of the node pair corresponding to the branch to be grown, and the variance of the pre-estimated errors comprises:

determining a mismatch error according to the relative pose of the node and the current node of the sub search tree;

determining an actual error variance according to the mismatch error;

14. A global relocation apparatus, comprising:

15. An electronic device, comprising: a memory and a processor;

the memory to store program instructions;

the processor, which is configured to call and execute the program instructions in the memory, to perform the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium; the computer program, when executed by a processor, implements the method of any one of claims 1-13.

17. A computer program product, comprising: a computer program; the computer program, when executed by a processor, implements the method of any one of claims 1-13.