WO2021057743A1 - 地图融合方法及装置、设备、存储介质 - Google Patents

地图融合方法及装置、设备、存储介质 Download PDF

Info

Publication number
WO2021057743A1
WO2021057743A1 PCT/CN2020/116926 CN2020116926W WO2021057743A1 WO 2021057743 A1 WO2021057743 A1 WO 2021057743A1 CN 2020116926 W CN2020116926 W CN 2020116926W WO 2021057743 A1 WO2021057743 A1 WO 2021057743A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
target
global
sampling point
coordinates
Prior art date
Application number
PCT/CN2020/116926
Other languages
English (en)
French (fr)
Inventor
金珂
杨宇尘
马标
李姬俊男
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2021057743A1 publication Critical patent/WO2021057743A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
    • G09B29/003—Maps
    • G09B29/005—Map projections or methods associated specifically therewith

Definitions

  • the embodiments of the present application relate to electronic technology, and relate to, but are not limited to, map fusion methods, devices, equipment, and storage media.
  • indoor environment maps can be established through visual information, and the need for map fusion is inevitably encountered in the process of constructing indoor environment maps.
  • map construction after multiple data collections, multi-person or multi-machine collaboration to complete map construction it is necessary to synthesize the local maps corresponding to each local area into a global map.
  • the synthesized global map has a large fusion error, and the consistency between multiple local maps after synthesis is low.
  • the map fusion method, device, device, and storage medium provided by the embodiments of the present application can reduce map fusion errors and improve the consistency problem of the global map obtained after fusion of multiple local maps.
  • the technical solutions of the embodiments of the present application are implemented as follows:
  • the map fusion method provided by the embodiment of the present application includes: acquiring a first map set and a second map, the first map set includes one or more different first maps, and the coordinate system of each first map The coordinate system of the second map is different; the coordinate system of the second map is used as the global coordinate system, and the local coordinates of the first sampling point in each of the first maps are converted into the global coordinate system, Obtain the first global coordinates of the first sampling point; fuse the first global coordinates of the first sampling point into the second map to obtain an initial global map; compare the first global coordinates in the initial global map The first global coordinate of the sampling point is optimized to obtain the target global map.
  • the map fusion device includes: a map acquisition module configured to acquire a first map set and a second map, the first map set includes one or more different first maps, each The coordinate system of the first map is different from the coordinate system of the second map; the coordinate conversion module is configured to take the coordinate system of the second map as the global coordinate system, and convert the first sampling point in each of the first map Transform the local coordinates of the first sampling point into the global coordinate system to obtain the first global coordinate of the first sampling point; a map fusion module configured to fuse the first global coordinate of the first sampling point to the second map In, the initial global map is obtained; the map optimization module is configured to optimize the first global coordinates of the first sampling point in the initial global map to obtain the target global map.
  • the electronic device provided by the embodiment of the present application includes a memory and a processor.
  • the memory stores a computer program that can run on the processor.
  • the processor implements the steps in the map fusion method when the program is executed.
  • the computer-readable storage medium provided by the embodiment of the present application has a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned map fusion method are realized.
  • the first global coordinates of the first sampling points in one or more different first maps are merged into the second map, and after the initial global map is obtained, the coordinates of the first sampling points in the initial global map
  • the first global coordinate is optimized to obtain the target global map; in this way, the map fusion error can be reduced, and the consistency problem of the initial global map can be improved.
  • FIG. 1A is a schematic diagram of the implementation process of a map fusion method according to an embodiment of this application;
  • FIG. 1B is a schematic diagram of fusing multiple first maps into a second map according to an embodiment of the application
  • FIG. 2 is a schematic diagram of determining local coordinates of multiple target points according to an embodiment of the application
  • FIG. 3 is a schematic diagram of a feature point matching pair according to an embodiment of the application.
  • 5A is a schematic diagram of the composition structure of a map fusion device according to an embodiment of the application.
  • 5B is a schematic diagram of the composition structure of another map fusion device according to an embodiment of the application.
  • FIG. 6 is a schematic diagram of a hardware entity of an electronic device according to an embodiment of the application.
  • the embodiments of this application provide a map fusion method, which can be applied to electronic devices, which can be mobile phones, tablet computers, notebook computers, desktop computers, robots, drones, or servers with information processing capabilities. equipment.
  • the functions implemented by the map fusion method can be implemented by invoking the program code by the processor in the electronic device.
  • the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.
  • FIG. 1A is a schematic diagram of the implementation process of the map fusion method according to the embodiment of the application. As shown in FIG. 1A, the method may at least include the following steps S101 to S104:
  • Step S101 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different.
  • the multiple different first maps can be local maps collected by multiple different electronic devices in different areas through built-in image acquisition modules, or they can also be local maps collected by one electronic device in different areas through built-in images.
  • these electronic devices may send the collected first map in the form of crowdsourcing to the electronic device implementing the map fusion method.
  • the second map is actually a partial map, but compared with each first map, the possible collection area is different.
  • the type of the coordinate system of the second map is not limited, and the coordinate system of the map may be a coordinate system suitable for a certain application scenario.
  • the coordinate system of the second map is a world coordinate system.
  • the coordinate system of the map can also be a custom coordinate system.
  • Step S102 Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the coordinate system of the first sampling point.
  • the first global coordinate Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the coordinate system of the first sampling point.
  • the first global coordinate Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the coordinate system of the first sampling point.
  • the first global coordinate Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the coordinate system of the first sampling point. The first global coordinate.
  • the local coordinates of the first sampling points that do not match the second map in the first map can be converted into the first global coordinates; when step S103 is performed, these first sampling points that do not match The first global coordinates of is merged into the second map.
  • the first global coordinates of the sampling points where the image features of the sampling points in the first map do not match the image features in the second map are added to the second map. That is, the first global coordinates of the sampling points of the area that does not overlap with the second map in the first map are added to the second map.
  • the electronic device may also merge the first global coordinates of all the first sampling points in the first map into the second map. It should be noted that the first global coordinate of each first sampling point is different. The first sampling point does not specifically refer to a certain sampling point in the first map, but to distinguish it from the second sampling point. That is, the first sampling point generally refers to the sampling point in the first map.
  • Step S103 fusing the first global coordinates of the first sampling point into the second map to obtain an initial global map.
  • the first global coordinates of the multiple first sampling points may be added to the second map.
  • the electronic device may also add the image features and the first global coordinates of the multiple first sampling points to the second map.
  • Step S104 optimizing the first global coordinates of the first sampling point in the initial global map to obtain a target global map.
  • the purpose of optimization is to obtain target global coordinates that are closer to the actual coordinates of the first sampling point, so as to reduce the cumulative error introduced when determining the first global coordinates of the first sampling point, thereby reducing Fusion errors are obtained to obtain a global map of the target.
  • a local map refers to a map corresponding to a certain floor or a room on a certain floor
  • a global map refers to a map corresponding to the office building.
  • the electronic device after the electronic device merges the first global coordinates of the first sampling point in each first map into the second map, it does not directly use the obtained initial global map as the target global map of the final application, but Yes, optimize the first global coordinate of the first sampling point in the initial global map; in this way, the fusion error can be reduced, and the obtained first global coordinate of the first sampling point is closer to the actual coordinates.
  • the embodiment of the present application further provides a map fusion method, and the method may include the following steps S201 to S204:
  • Step S201 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S202 taking the coordinate system of the second map as the global coordinate system, sequentially converting the local coordinates of the first sampling points in the first map that meet the conditions in the first map set into the first global coordinates;
  • step S203 the first global coordinates of the first sampling points in the first map meeting the conditions in the first map set are sequentially merged into the current second map to obtain the initial global map.
  • the electronic device can implement step S202 and step S203 through step S302 to step S305 in the following embodiment, that is, preferentially convert the local coordinates of the first sampling point in the first map matching the second map in the first map set It is the first global coordinate and merged into the current second map.
  • the first map with the largest number of matching pairs with the second map is merged into the current second map first, that is, the first map with the most overlapping area with the second map is merged into the current second map first.
  • Step S204 optimizing the first global coordinates of the first sampling point in the initial global map to obtain a target global map.
  • the first map that meets the conditions is preferentially fused, which not only improves the fusion accuracy, but also avoids the failure of fusion, thereby improving the efficiency of map fusion.
  • the embodiment of the present application further provides a map fusion method, the method at least includes the following steps S301 to S306:
  • Step S301 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S302 Determine a first map matching the second map in the first map set as a target local map.
  • the electronic device can implement step S302 through step S402 to step S404 in the following embodiment; or, the electronic device can also implement step S302 through step S502 to step S504 in the following embodiment.
  • Step S303 Using the coordinate system of the second map as the global coordinate system, the local coordinates of the first sampling point in the target local map are converted into the first global coordinates.
  • the electronic device may implement step S303 through step S311 to step S313, or step S405, or step S502 to step S504 in the following embodiments.
  • Step S304 fusing the first local coordinates of the first sampling point in the target local map into the second map to obtain a fused second map
  • Step S305 From the remaining first maps in the first map set, determine a new target local map that matches the fused second map, so as to sample the new target local map first.
  • the first global coordinates of the points are merged into the merged second map until each of the first maps is merged into the second map to obtain the initial global map.
  • the first map included in the first map set 10 is map 11, map 12, and map 13.
  • the first map set 10 and the second map 14 matches the first map as the map 12.
  • the map 12 is used as the target local map and merged into the second map 14, and the resulting merged second map 14 is the map 141;
  • the second map fusion determine from the first map set the new target local map that matches the map 141 (that is, the current second map) as map 11.
  • map 11 is merged into map 141, and the result is The merged second map is the map 142; finally, the map 13 is merged into the map 142 (that is, the current second map) to obtain the initial global map 143.
  • step S306 the first global coordinate of the first sampling point in the initial global map is optimized to reduce the fusion error and obtain the target global map.
  • the first map to be fused ie the target local map
  • the map fusion are determined, and these two actions are performed alternately, that is, the new target local map is determined on the basis of the currently obtained second map
  • the electronic device can find the target local map with more overlapping areas with the currently obtained second map from the first map set, which determines the accuracy of the first global coordinates of the first sampling point in the target local map It is advantageous and can reduce the fusion error and improve the map fusion accuracy.
  • the electronic device can implement the above step S303 through the following steps S311 to S313 (that is, the coordinate system of the second map is used as the global coordinate system, and the first sampling point in the target local map is Steps of converting local coordinates to the first global coordinates):
  • Step S311 From the second sampling points of the second map, determine a target point that matches the image feature of the first sampling point in the target local map;
  • Step S312 according to the local coordinates of the multiple first sampling points in the target local map and the global coordinates of the target points respectively corresponding to the multiple first sampling points, determine that the target local map is relative to the second The first coordinate conversion relationship of the map.
  • the electronic device may determine the correlation with the multiple first sampling points according to the local coordinates of the multiple first sampling points and the global coordinates of the target points respectively corresponding to the multiple first sampling points. Respectively corresponding to the local coordinates of the target point in the target local map; the first coordinate conversion relationship is determined according to the global coordinates and the local coordinates of each target point.
  • the first coordinate conversion relationship includes the rotation relationship of the target local map relative to the second map
  • the electronic device may determine, according to the rotation relationship, that the lens of the image acquisition module is relative to the target local map when collecting the target local map.
  • the electronic device can accurately determine the relationship with the three first sampling points based on the local coordinates of at least three first sampling points and the global coordinates of target points matching the three first sampling points.
  • the local coordinates of the matched target points respectively.
  • point O is the origin of the coordinate system of the target local map
  • the multiple target points are the three uppercase A, B, and C points shown in Figure 2.
  • the first sampling point matching point A is lowercase point a
  • the first sampling point matching point B is lowercase point b
  • the first sampling point matching point C is lowercase point c.
  • ⁇ a,b> refers to ⁇ aOb
  • ⁇ a,c> refers to ⁇ aOc
  • ⁇ b,c> refers to ⁇ bOc.
  • OA, OB, and OC are the distances between the origin of the coordinate system of the target local map and the target points A, B, and C, respectively.
  • the direction is from point O to point a;
  • the direction is from point O to point b;
  • the direction is from point O to point c.
  • Step S313 According to the first coordinate conversion relationship, the local coordinates of the first sampling point in the target local map are converted into the first global coordinates.
  • step S313 it can convert the local coordinates of each first sampling point in the target local map to the first global coordinates; it can also convert the local coordinates of part of the first sampling points in the target local map to the first global coordinates.
  • a global coordinate for example, converting the local coordinates of other first sampling points in the target local map except the first sampling point matching the target point into the first global coordinates.
  • the embodiment of the present application further provides a map fusion method, and the method may include the following steps S401 to S410:
  • Step S401 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S402 Match the image features of the first sampling points in each of the first map with the image features of the second sampling points in the second map to obtain a first match corresponding to the first map. Pair collection.
  • the electronic device can obtain the first matching pair set corresponding to each first map.
  • each first set of matching pairs can be obtained through step S402.
  • the electronic device may determine the first matching pair set corresponding to the first map through the following steps, namely: determining the image feature of the i-th first sampling point in the first map and each second The similarity between the sampling points is obtained, and i is an integer greater than 0; the second sampling point in the similarity set whose similarity satisfies the second condition is determined as the target point; each target is The points are associated with the corresponding first sampling points to obtain a first matching pair set corresponding to the first map.
  • the matching pairs included in the first matching pair set are the first sampling point and the second sampling point (that is, the target point) matching the first sampling point.
  • the similarity refers to the similarity between the image features of two sampling points.
  • the similarity can be characterized by Euclidean distance, Hamming distance, or cosine similarity.
  • the second condition can be various, which is not limited in this application.
  • the second condition is that the similarity is less than the third threshold.
  • the second condition is that the similarity in the similarity set is the smallest.
  • Step S403 Determine the first matching pair set that meets the first condition as the first target matching pair set.
  • the first condition can also be varied.
  • the first condition is that the number of matching pairs is greater than the fourth threshold.
  • the first condition is that the number of matching pairs is the largest.
  • Step S404 Determine the first map corresponding to the first target matching pair set as the target local map
  • Step S405 Determine the first coordinate conversion relationship of the target local map relative to the second map according to the local coordinates of the multiple first sampling points in the first target matching pair set and the global coordinates of the corresponding target points ;
  • Step S407 using the coordinate system of the second map as the global coordinate system, and converting the local coordinates of the first sampling point in the target local map into the first global coordinates according to the first coordinate conversion relationship;
  • Step S408 fusing the first local coordinates of the first sampling point in the target local map into the second map to obtain a fused second map
  • Step S409 From the remaining first maps in the first map set, determine a new target local map that matches the fused second map, so as to sample the first local map from the new target local map.
  • the first global coordinates of the points are merged into the merged second map until each of the first maps is merged into the second map to obtain the initial global map;
  • Step S410 optimizing the first global coordinates of the first sampling point in the initial global map to reduce the fusion error and obtain the target global map.
  • a target local map that matches the second map is selected from the first map set; in this way, the electronic device It is possible to find a target local map that is more matched with the second map, because according to the image characteristics, a better matching accuracy can be obtained; this is also more conducive to improving the fusion accuracy.
  • the embodiment of the present application further provides a map fusion method, and the method may include the following steps S501 to S508:
  • Step S501 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S502 According to the iteration strategy, the local coordinates of each first sampling point of the nth first map in the first map set are matched with the global coordinates of the multiple second sampling points in the second map. The matching result is obtained, and n is an integer greater than 0.
  • each first map includes the local coordinates of the first sampling point
  • the second map includes the global coordinates of the second sampling point, but these maps do not include the image features of the sampling points; in this way, the map fusion method can be applied
  • the electronic device can also realize map fusion.
  • the image features generally occupy a relatively large storage space.
  • an image feature is a feature descriptor.
  • the feature descriptor of each sampling point has 256 bytes, which requires the electronic device to allocate at least 256 bytes of storage space for each sampling point to store the feature description child.
  • the target global map does not include the image features of the sampling points; in this way, the data volume of the target global map can be greatly reduced, thereby saving storage resources consumed by the electronic device to store the target global map.
  • Step S503 in a case where the matching result indicates that the matching is successful, determine the n-th first map as the target local map;
  • Step S504 in the case where the matching result indicates that the matching fails, continue to match the local coordinates of each first sampling point in the next first map with the global coordinates of the multiple second sampling points until all Until the target local map is determined in the first map set, step S505 is executed;
  • Step S505 Using the coordinate system of the second map as the global coordinate system, convert the local coordinates of the first sampling point in the target local map to the first global coordinates;
  • Step S506 fusing the first local coordinates of the first sampling point in the target local map into the second map to obtain a fused second map
  • Step S507 From the remaining first maps in the first map set, determine a new target local map that matches the fused second map, so as to sample the new target local map first.
  • the first global coordinates of the points are merged into the merged second map until each of the first maps is merged into the second map to obtain the initial global map;
  • step S508 the first global coordinate of the first sampling point in the initial global map is optimized to reduce the fusion error and obtain the target global map.
  • the embodiment of the present application further provides a map fusion method, and the method may include the following steps S601 to S614:
  • Step S601 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S602 Select an initial target point matching each first sampling point in the nth first map from a plurality of second sampling points in the second map.
  • the initial coordinate conversion relationship of the first map relative to the second map may be set; then, according to the local coordinates of the first sampling point in the first map and the initial conversion relationship, the first map The first sampling point of is matched with the plurality of second sampling points, so that an initial target point matching the first sampling point in the first map is selected from the plurality of second sampling points.
  • the electronic device may select the initial target point through steps S702 to S704 in the following embodiments.
  • step S602 is used to select a second sampling point that may match the first sampling point in the first map.
  • the initial target point may not be a point that actually matches the first sampling point. Therefore, the following step S603 is required. Go to step S610 to further determine whether the initial target point is a point that truly matches the first sampling point.
  • Step S603 According to the local coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point, determine the position of the nth first map relative to the second map. Two-coordinate conversion relationship.
  • step S603 it can construct an error function based on the local coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point; then, the current optimal value can be solved by the least squares method.
  • the second coordinate conversion relationship
  • the global coordinates of a point are expressed by q l , then, the following formula (7) can be listed:
  • E(R, T) is the error function
  • R and T are respectively the second rotation relationship and the second translation relationship in the second coordinate transformation relationship to be solved. Then, the optimal solution of R and T in equation (7) can be solved by the least square method.
  • Step S604 Determine a matching error according to the second coordinate conversion relationship, the local coordinates of each first sampling point in the nth first map, and the global coordinates of the corresponding initial target point.
  • step S604 it can determine the matching error through step S706 and step S707 in the following embodiment.
  • Step S605 counting the number of times of determining matching errors.
  • step S606 it is determined whether the number of times is greater than the second threshold; if so, step S607 is executed; otherwise, step S608 is executed.
  • the nth first map can be replaced with the n+1th (ie, the next) first map, and the contents similar to steps S602 to 606 can be executed again.
  • Step S607 Generate a matching result that characterizes the failure of the matching, and continue to select an initial target point that matches each first sampling point in the next first map from the plurality of second sampling points, until a character that characterizes the successful matching is generated Until the matching result, the process proceeds to step S610.
  • Step S608 Determine whether the matching error is greater than the first threshold; if so, return to step S602, reselect the initial target point, and re-determine the matching error; otherwise, perform step S609.
  • Step S609 Generate a matching result that characterizes the success of the matching.
  • Step S610 when the matching result indicates that the matching is successful, determine the nth first map as the target local map, and determine the second when the determined matching error is less than or equal to the first threshold.
  • the coordinate conversion relationship is determined as the first coordinate conversion relationship between the target local map and the second map.
  • the matching error is greater than the first threshold, it means that the currently selected initial target point is not a point that matches the first sampling point in the current first map, and they do not refer to the same location point in the physical space. Or a similar location point.
  • the initial target point selected in the current iteration is a point that truly matches the first sampling point in the current first map.
  • the second coordinate conversion relationship obtained in the current iteration can be determined as the first coordinate conversion relationship
  • Step S611 taking the coordinate system of the second map as the global coordinate system, and converting the local coordinates of the first sampling point in the target local map into the first global coordinates according to the first coordinate conversion relationship.
  • step S612 the first local coordinates of the first sampling point in the target local map are merged into the second map to obtain a merged second map.
  • Step S613 From the remaining first maps in the first map set, determine a new target local map that matches the fused second map, so as to sample the new target local map as the first The first global coordinates of the points are merged into the merged second map until each of the first maps is merged into the second map to obtain the initial global map.
  • step S614 the first global coordinate of the first sampling point in the initial global map is optimized to reduce the fusion error and obtain the target global map.
  • the embodiment of the present application further provides a map fusion method, and the method may include the following steps S701 to S717:
  • Step S701 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S702 Obtain a third coordinate conversion relationship of the n-th first map relative to the second map; in implementation, the third coordinate conversion relationship may be set to an initial value;
  • Step S703 Determine the second global coordinates of the j-th first sampling point according to the third coordinate conversion relationship and the local coordinates of the j-th first sampling point in the n-th first map, where j is An integer greater than 0;
  • Step S704 Match the second global coordinates with the global coordinates of the multiple second sampling points to obtain an initial target point that matches the j-th first sampling point.
  • step S704 it can determine the distance between the second global coordinate of the j-th first sampling point and the global coordinate of each second sampling point (for example, Euclidean distance); The second sampling point closest to the j-th first sampling point is determined as the initial target point, or the second sampling point whose distance is less than or equal to the distance threshold is determined as the initial target point.
  • the distance between the second global coordinate of the j-th first sampling point and the global coordinate of each second sampling point for example, Euclidean distance
  • the second sampling point closest to the j-th first sampling point is determined as the initial target point, or the second sampling point whose distance is less than or equal to the distance threshold is determined as the initial target point.
  • Step S705 According to the local coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point, determine the position of the nth first map relative to the second map. Two-coordinate conversion relationship;
  • Step S706 Determine the third global coordinate of the j-th first sampling point according to the second coordinate conversion relationship and the local coordinates of the j-th first sampling point in the n-th first map, where j is greater than 0 Integer
  • Step S707 Determine the matching error according to the third global coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point.
  • the electronic device When the electronic device implements step S707, it may first determine the distance between the third global coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point (for example, Euclidean distance, etc.) ; According to each of the distances, the matching error is determined.
  • the distance between the third global coordinates of each first sampling point in the nth first map and the global coordinates of the corresponding initial target point for example, Euclidean distance, etc.
  • the electronic device may determine the average distance between the multiple first sampling points and the matched initial target point as the matching error.
  • the matching error d can be obtained:
  • Step S708 counting and determining the number of matching errors
  • Step S709 Determine whether the number of times is greater than a second threshold; if yes, go to step S710; otherwise, go to step S711;
  • Step S710 Generate a matching result that characterizes the failure of the matching, and return to step S702 to continue to obtain the third coordinate conversion relationship between the next first map and the second map, until a matching result that characterizes the successful matching is generated, and then go to step S713;
  • Step S711 Determine whether the matching error is greater than the first threshold; if so, use the second coordinate conversion relationship as the third coordinate conversion relationship, and then return to step S703 to reselect the initial target point; otherwise, execute Step S712.
  • the matching error is greater than the first threshold, it indicates that the acquired third coordinate conversion relationship does not conform to reality.
  • the obtained initial target point is not a point that really matches the first sampling point.
  • the second coordinate conversion relationship can be used as the third coordinate conversion relationship, and steps S703 to S710 are re-executed until Until the matching error is less than the first threshold, step S712 is executed.
  • Step S712 generating a matching result that characterizes the successful matching
  • Step S713 In a case where the matching result indicates that the matching is successful, the n-th first map is determined as the target local map.
  • the matching result that characterizes the successful matching is obtained by the electronic device matching the local coordinates of each first sampling point in the next first map with the global coordinates of the multiple second sampling points As a result of the matching, the next first map is determined as the target local map at this time. That is, when the current matching result indicates that the matching is successful, the first map currently being matched is determined as the target local map.
  • Step S714 Using the coordinate system of the second map as the global coordinate system, convert the local coordinates of the first sampling point in the target local map to the first global coordinates;
  • Step S715 fusing the first local coordinates of the first sampling point in the target local map into the second map to obtain a fused second map
  • Step S716 From the remaining first maps in the first map set, determine a new target local map that matches the fused second map, so as to sample the new target local map first.
  • the first global coordinates of the points are merged into the merged second map until each of the first maps is merged into the second map to obtain the initial global map;
  • step S717 the first global coordinates of the first sampling points in the initial global map are optimized to reduce the fusion error and obtain the target global map.
  • the method may include the following steps S801 to S806:
  • Step S801 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S802 Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the coordinate system of the first sampling point.
  • Step S803 fusing the first global coordinates of the first sampling point into the second map to obtain an initial global map
  • Step S804 Obtain the determined first target matching pair set when each of the first maps is fused into the current second map, and the first target matching pair set includes the i-th target in the first map.
  • i is an integer greater than 0.
  • each first map is fused based on the previous first map being fused to the second map, that is to say, the second map after the previous fusion can be understood as the current second map .
  • the current second map to which the map 12 is fused is the second map 14
  • the fused second map 14 is the map 141
  • the current second map to which the map 11 is fused is Map 141.
  • the electronic device may determine the first target matching pair set similar to the above-mentioned step S402 and step S403, here, it can be directly obtained.
  • Step S805 optimizing the first global coordinate of each first sampling point in the set for each of the first target matching to obtain the target global coordinate of the corresponding first sampling point;
  • Step S806 Update the target global coordinates of each first sampling point to the initial global map to obtain the target global map.
  • coordinate optimization is performed on the first sampling point with the first global coordinate in the first target matching set. In this way, the calculation amount for performing step S805 can be reduced, and the efficiency of obtaining the global map of the target can be improved.
  • step S805 can be implemented at least through the following steps S811 to S812:
  • Step S811 Determine the reprojection error of the corresponding first sampling point according to the first global coordinate of each first sampling point in each first target matching pair set.
  • determining the reprojection error of the first sampling point in the first target matching pair set can be achieved by the following steps:
  • the coordinate conversion relationship of n is an integer greater than 0;
  • the depth information of the k-th first sampling point is obtained, and the k-th first sampling point is the first target matching pair set corresponding to the n-th first map
  • the projection matrix, the first coordinate conversion relationship, the first global coordinates of the k-th first sampling point, and the depth information determine the k-th first sample
  • the point's reprojection error, k is an integer greater than 0.
  • Step S812 Adjust the first global coordinates of each first sampling point in each first target matching pair set, so that the reprojection error of each first sampling point meets the third condition, thereby obtaining the corresponding first sampling The target global coordinates of the point.
  • the first global coordinate of each first sampling point can be adjusted at the same time through step S812, so that the reprojection error of each first sampling point meets the third condition.
  • the third condition may be, for example, that the reprojection error function value determined by the reprojection error of each first sampling point is smaller than the second threshold, so that the target global coordinates of each first sampling point can be obtained.
  • the electronic device may implement step S812 in this way, that is, determine the reprojection error function value according to the reprojection error of each first sampling point in the set of each of the first target matching pairs;
  • the first global coordinate of the first sampling point corresponding to the projection error is used to minimize the reprojection error function value to obtain the target global coordinate of each first sampling point in each first target matching pair set.
  • the reprojection error of the corresponding first sampling point is determined according to the first global coordinate of each first sampling point in each first target matching pair set, It can be implemented at least through the following steps S821 to S824:
  • Step S821 Obtain the projection matrix corresponding to the nth first map in the first map set
  • Step S822 Obtain a first coordinate conversion relationship, where the first coordinate conversion relationship is the coordinate conversion relationship of the n-th first map relative to the current second map when it is fused, and n is an integer greater than 0;
  • Step S823 Obtain depth information of the k-th first sampling point, where the k-th first sampling point is the first sampling point in the first target matching pair set corresponding to the n-th first map;
  • Step S824 Determine the reprojection of the kth first sampling point according to the projection matrix, the first coordinate conversion relationship, the first global coordinates of the kth first sampling point, and the depth information Error, k is an integer greater than 0.
  • the electronic device can implement step S824 as follows: determine the k-th first sampling point according to the projection matrix, the first coordinate conversion relationship, the first global coordinates of the k-th first sampling point, and the depth information.
  • the projection coordinates of the sampling point in the current second map; according to the projection coordinates and the pixel coordinates of the target point matching the k-th first sampling point, the repetition of the k-th first sampling point is determined Projection error.
  • s k refers to the depth information of the k-th first sampling point
  • K represents the projection matrix
  • exp( ⁇ ) represents the first coordinate conversion relationship
  • the electronic device can iteratively adjust the first global coordinates of the first sampling point corresponding to each of the re-projection errors, so as to minimize the value of the re-projection error function, and obtain each of the first target matches.
  • Target global coordinates for each first sampling point in the set can be implemented through step S910 to step S914 in the following embodiment.
  • An embodiment of the present application is another map fusion method, and the method may include the following steps S901 to S916:
  • Step S901 Obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as the coordinate system of the second map. different;
  • Step S902 Using the coordinate system of the second map as the global coordinate system, transform the local coordinates of the first sampling point in each of the first maps into the global coordinate system to obtain the First global coordinate
  • Step S903 fusing the first global coordinates of the first sampling point into the second map to obtain an initial global map
  • Step S904 Obtain the first target matching pair set determined when each of the first maps is fused into the current second map, and the first target matching pair set includes the i-th target set in the first map. A first sampling point and a second sampling point matching the i-th first sampling point in the current second map, where i is an integer greater than 0;
  • Step S905 Obtain the projection matrix corresponding to the nth first map in the first map set
  • Step S906 Obtain a first coordinate conversion relationship, where the first coordinate conversion relationship is the coordinate conversion relationship of the nth first map relative to the current second map when it is fused, and n is an integer greater than 0;
  • Step S907 Obtain depth information of the k-th first sampling point, where the k-th first sampling point is the first sampling point in the first target matching pair set corresponding to the n-th first map;
  • Step S908 Determine the reprojection of the kth first sampling point according to the projection matrix, the first coordinate conversion relationship, the first global coordinates of the kth first sampling point, and the depth information Error, k is an integer greater than 0;
  • Step S909 Determine a reprojection error function value according to the reprojection error of each first sampling point in the set of each of the first target matching pairs;
  • Step S910 adjusting each of the first coordinate conversion relationships to obtain a corresponding fourth coordinate conversion relationship
  • Step S911 according to the conversion relationship between the local coordinates of the k-th first sampling point and the fourth coordinate, re-determine the first global coordinates of the k-th first sampling point;
  • Step S912 according to the depth information of the k-th first sampling point and the newly determined first global coordinates, the fourth coordinate conversion relationship and the projection matrix, re-determine the value of the k-th first sampling point Reprojection error
  • Step S913 re-determine the re-projection error function value according to the re-determined re-projection error of each first sampling point in the set of each of the first target matching pairs;
  • Step S914 if the re-determined reprojection error function value is less than a second threshold, determine the fourth global coordinate as the target global coordinate;
  • Step S915 if the re-determined re-projection error function value is greater than or equal to the second threshold, continue to adjust the fourth coordinate conversion relationship to re-determine the re-projection error function value until the currently determined re-projection error function value is less than Up to the second threshold, determining the currently determined first global coordinate as the target global coordinate;
  • step S916 the target global coordinates of each first sampling point are updated to the initial global map to obtain the target global map.
  • each first coordinate conversion relationship is adjusted iteratively to find the optimal target global coordinates; in this way, the optimal target global coordinates of the first global coordinates can be determined more quickly, thereby improving map fusion Accuracy.
  • the electronic device may implement the pre-construction of the second map through the following steps S111 to S115:
  • Step S111 Determine the global coordinates of at least one first sample point according to the local coordinates and image characteristics of the first sample point in the multiple sample images.
  • the image acquisition module can be used to acquire sample images at a specific frame rate.
  • a monocular camera is used to collect red, green, and blue (Red, Green, Blue, RGB) images at a fixed frame rate.
  • the multiple sample images can also be obtained from a sample image library collected in advance.
  • the image features and local coordinates of the first sample point can be obtained, but the global coordinates of the first sample point are unknown.
  • multiple sample images can be processed through a three-dimensional reconstruction method to obtain the global coordinates of the first sample point.
  • the structure from motion (SFM) method is used to initialize multiple sample images to obtain the global coordinates of each sample point.
  • Step S112 Determine a first data set according to the global coordinates and image features of each of the first sample points; that is, the first data set includes the global coordinates and corresponding image features of each of the first sample points;
  • Step S113 Determine a second data set corresponding to the m-th other sample image according to the acquired local coordinates and image features of the second sample point in the m-th other sample image, where m is an integer greater than 0;
  • Step S114 Determine the global coordinates of the second sample point in the second data set according to the first data set.
  • step S111 the time complexity of determining the global coordinates of the first sample point in the multiple sample images through step S111 is relatively high. Therefore, after the global coordinates of the first sample point are obtained, step S114 is used to determine the global coordinates of the second sample point; in this way, the time cost for constructing the second map can be greatly reduced.
  • the global coordinates of the second sample point can be determined through steps S311 to S313 similar to those provided in the foregoing embodiment. Alternatively, the global coordinates of the second sample point are determined by similarly to steps S502 to S505.
  • Step S115 constructing the second map at least according to the global coordinates of each of the first sample points and the global coordinates of each of the second sample points.
  • the constructed second map is the second map that does not integrate any of the first maps in the foregoing embodiment.
  • the first sample point and the second sample point are called second sample points in the second map.
  • the constructed second map includes the global coordinates and image features of the second sampling point.
  • the global coordinates of the first sample point when constructing a map, first obtain the global coordinates of the first sample point through multiple sample images, and then according to the global coordinates of each first sample point and the obtained other sample images The local coordinates and image features of the second sample point are determined, and the global coordinates of the second sample point in the other sample images are determined to obtain the second data set; in this way, the global coordinates of the sample points in the other sample images can be quickly obtained, thereby Reduce the construction cost of the second map.
  • the global coordinates of at least one first sample point are determined according to the local coordinates and image characteristics of the first sample point in the multiple sample images, which can be implemented through the following steps S121 to S123 :
  • Step S121 According to the local coordinates and image characteristics of each of the first sample points, select the first target image and the second target image that meet the fourth condition from the multiple sample images.
  • the selected first target image and second target image are generally two sample images with relatively large disparity, so that the accuracy of determining the global coordinates of the sample points in the first target image or the second target image can be improved. In turn, it is beneficial to obtain better map fusion accuracy in the future.
  • the multiple sample images are matched in pairs to obtain the second matching pair set of each pair of sample images; and the first matching is eliminated Obtain a third matching pair set for matching pairs in the set that do not meet the fourth condition; from each of the third matching pair sets, select a second target matching pair set whose number of matching pairs meets the fourth condition; The two sample images corresponding to the second target matching pair set are determined as the first target image and the second target image.
  • Step S122 Determine a fifth coordinate conversion relationship between the first target image and the second target image.
  • the four-point method in the Random Sample Consensus (RANSAC) algorithm can be used to process the first target image and the second target image, and calculate the homography matrix to obtain the fifth Coordinate transformation relationship (including rotation relationship and translation relationship).
  • Step S123 Determine the global coordinates of the first sample point in the first target image according to the fifth coordinate conversion relationship and the local coordinates of the first sample point in the first target image.
  • the sample point in the first target sample image and the sample point of the matched second target sample image are actually at the same location point, so here, the global coordinates of the sample point in any one of the two target sample images is determined That's it.
  • the embodiments of the present application implement an indoor map fusion technology based on sparse point clouds, which can achieve the goals of fusion of multiple local maps (including the first map and the second map) and map update.
  • This solution can support the needs of multiple indoor local map synthesis with repeated areas.
  • the collection of local maps can be collected disorderly in the form of crowdsourcing.
  • the solution can support daily tasks such as map fusion, map update, and multi-person map building.
  • the synthesized map has high accuracy and strong robustness.
  • the local map is a sparse point cloud map constructed by collecting RGB image information through a monocular camera and extracting image features from it.
  • the extracted image feature is the ORB descriptor information of FAST corner points (ie, feature points), and the three-dimensional coordinate information of each corner point is determined.
  • the specific technical steps for constructing the second map include at least the following steps S11 to S15:
  • Step S11 using a monocular camera to collect RGB images at a fixed frame rate
  • Step S12 extract feature information in the image in real time during the acquisition process
  • Step S13 after a certain number of images are collected, the SFM method is used to initialize;
  • Step S14 after the initialization is completed, the three-dimensional coordinates of the subsequent image feature points (that is, the global coordinates of the second sample point) are calculated through the PnP (Perspective-n-Point) algorithm to obtain the second map;
  • PnP Perspective-n-Point
  • step S15 the second map and its corresponding image feature information are stored, and serialized and stored locally.
  • Feature extraction is a process of interpretation and annotation of RGB images.
  • FAST corner points are extracted from the RGB image, and the number of extraction is fixed to 150, which is used for image tracking; and the ORB descriptor is extracted for the corner points for feature point descriptor matching.
  • 150 is an empirical value, and the number of extracted corner points is not limited in this application. The number of corner points is too small, the tracking failure rate is high, and the number of corner points is too much, which affects the algorithm efficiency.
  • the SFM method is used for initialization, and the following explanation is given here.
  • the relative rotation and translation of the images are initialized by the SFM method, and the three-dimensional coordinates of the feature points (ie, the global coordinates of the first sample point) are obtained.
  • the SFM algorithm includes at least the following steps S131 to S139:
  • Step S131 Perform pairwise matching on a certain number of images, and establish a matching relationship between image feature points using a method of Euclidean distance judgment;
  • Step S132 the matching pairs are eliminated, the elimination method adopts the RANSAC eight-point method to calculate the basic matrix, and the matching pairs that do not satisfy the basic matrix are selected to be eliminated;
  • Step S133 After the matching relationship is established, a tracking list is generated, and the tracking list refers to a collection of image names of points with the same name;
  • Step S134 removing invalid matches in the tracking list
  • Step S135 searching for an initial image pair (ie, the first target image and the second target image).
  • the purpose is to find the image pair with the largest camera baseline.
  • the four-point method of the RANSAC algorithm is used to calculate the homography matrix, and the matching points that satisfy the homography matrix become internal Point, the unsatisfied become the outside point. Then, find the image pair with the smallest proportion of interior points.
  • Step S136 searching for the relative rotation and translation of the initial image pair, the method is to calculate the essential matrix by the RANSAC eight-point method, and obtain the relative rotation and translation between the image pair by SVD decomposition of the essential matrix;
  • Step S137 Obtain the three-dimensional coordinates of the feature points in the initialization image pair through triangulation calculation
  • Step S138 Repeating step S136 and step S137 on other images, the relative rotation and translation of all images and the three-dimensional coordinates of the feature points can be obtained;
  • step S139 the rotation and translation between the obtained images and the three-dimensional coordinates of the characteristic points are optimized through the beam adjustment method. This is a non-linear optimization process, the purpose is to reduce the error of the SFM results.
  • an offline map based on the sparse point cloud can be constructed, that is, the second map, which stores the sparse point cloud and its image feature information (including three-dimensional coordinates and descriptor information) in a binary format to the local, During the visual positioning process, the map will be loaded and used.
  • the map update part is mainly used to match the corresponding sparse point clouds in the two local maps through the descriptor to form a point cloud matching pair; and then use the PnP algorithm to solve the current first map relative to the second map After the precise pose of the first map is rotated and translated, it is merged into the second map to achieve the purpose of map update.
  • the specific technical steps include at least the following steps S21 to S26:
  • Step S21 load the constructed second map and the first map, and use the second map coordinate system as the global coordinate system;
  • Step S22 in the first map, match corresponding points in the second map through the descriptor to form a point cloud matching pair;
  • Step S23 After finding a sufficient number (for example, a specific number) of matching pairs, the PnP algorithm is used to solve the precise pose of the sparse point cloud in the first map in the global coordinate system;
  • Step S24 updating all the sparse point clouds in the first map to the global coordinate system, and adding the sparse point cloud set of the first map to the sparse point cloud set of the second map;
  • Step S25 Repeat steps S22 to S24 to merge other local maps into the global coordinate system
  • step S26 an optimization method is used to optimize the posture of the local map after fusion, eliminate the accumulated error of the fusion of multiple local maps, and further improve the accuracy of map update.
  • step S22 in the first map, the corresponding points in the second map are matched by the descriptor to form a point cloud matching pair.
  • the specific steps of the algorithm include at least the following steps S221 to S224:
  • Step S222 Calculate F 1N and the M-th (initially 0) sparse point F 2M in the point cloud in the second map, and calculate the Euclidean distance d NM between the feature point descriptors;
  • step S224 the sparse point matching pairs in the first map and the second map are sorted as the algorithm output, and the algorithm ends.
  • step S23 the PnP algorithm is used to solve the precise pose of the sparse point cloud in the first map in the global coordinate system.
  • a preferred example is shown in Figure 3.
  • the algorithm steps are as follows:
  • step S22 it is judged that a matching pair sequence is formed in step S22 (in this example, the matching pair sequence is ⁇ F 0 , F 1 , F 2 ⁇ ). If the number of elements in the matching pair sequence is greater than TH 2 , then step S23 is performed, otherwise the algorithm ends, The local point cloud fusion failed.
  • the SolvePnP function in OpenCV is called to find the pose of the current camera in the map coordinate system.
  • the principle of the PnP algorithm is as follows:
  • the input of the PnP algorithm is the 3D points in the second map (that is, the second sampling point) and the 2D points obtained by the projection of these 3D points in the first map, and the output is the global coordinate system of the first map relative to the second map
  • the pose of the origin is transformed.
  • the PnP algorithm first obtains the 3D coordinates of the corresponding 2D point in the coordinate system of the first map according to the matched pair sequence, and then according to the coordinates of the 3D point in the global coordinate system and the 2D point in the coordinate system of the first map.
  • the 3D coordinates solve the global coordinates of the sparse points in the first map.
  • step S24 updating the sparse point cloud in the first map to the global coordinate system
  • the second local pose can be obtained from the global pose and the local pose of the sparse point cloud in the first map.
  • the rotation matrix and translation vector of the coordinate system relative to the global coordinate system.
  • step S26 using the optimization method to optimize the fusion of the local map pose, the following explanation is given here.
  • the scheme of minimizing the reprojection error is usually adopted, and each local map corresponds to a rotation and translation state, and the connection between the multiple local maps is constructed through the matching of sparse point clouds.
  • the sparse point matching pairs of the local maps have been obtained in step S22, and the rotation and translation of each local map in the global coordinate system are also calculated in step S23.
  • the pose and sparse point matching pairs of all local maps are comprehensively considered. Constructing the reprojection error function and then minimizing this cost function can optimize the fusion accuracy of the local map.
  • the observation value ⁇ p 1 , p 2 ⁇ is a set of feature matching pairs, which are projections of the same spatial point P.
  • p 1 belongs to the first map.
  • p 2 belongs to the second map, To convert p 1 into a projection point on the second map after the first global coordinates in the second map.
  • sk represents the depth information of the spatial point P k in the first map
  • K represents the projection matrix of the first map. Due to the local map pose error and the noise of the observation point, this equation has a reprojection error. Therefore, as shown in formula (14), the reprojection errors are summed to construct the least squares problem to find the best local map Pose, minimize the equation:
  • the pixel coordinates of the ⁇ k space point P k in the second map Indicates the reprojection error of the spatial point P k .
  • the projection error is the projection The pixel distance from the observation value p 2.
  • the Gauss-Newton method or Levenberg-Marquardt method can be used to solve this least squares problem, so that the optimized local map pose can be obtained.
  • map update scheme has high fusion accuracy and strong robustness.
  • the cumulative error of map fusion is reduced, and the consistency of the global map after synthesis is improved.
  • a high-precision and high-robust matching algorithm is used in combination with the three-dimensional information of visual features, and the result of map fusion improves the fusion accuracy compared with other indoor positioning methods.
  • the stored map is in the form of a sparse point cloud, which is equivalent to sparse sampling of the image, and the map size has a certain degree of compression compared with the traditional method.
  • the three-dimensional information of image features is fully excavated, and the goal of map fusion is achieved by combining optimization theory.
  • the embodiment of the present application collects the three-dimensional coordinates and descriptor information of the feature points in the visual image, and stores them as an offline map in the form of a sparse point cloud.
  • the embodiment of this application uses the descriptor matching method to find the matching pair of the current feature point in the sparse point cloud; and then uses the PnP algorithm to accurately calculate the position and posture of the current local map in the global coordinate system;
  • the optimization theory is combined to achieve the purpose of map fusion, forming a set of map fusion schemes with high fusion accuracy and strong robustness.
  • the solution supports map fusion in multi-person mapping scenarios and crowdsourced map updates, which not only ensures the stability of map fusion, but also improves the efficiency of building local maps.
  • the embodiment of the present application only serializes and stores the three-dimensional coordinate information of the sparse point cloud.
  • the embodiment of this application proposes an adjusted map fusion scheme, and the technical steps at least include the following steps S31 to S35:
  • Step S31 load the constructed second map and the first map, and use the second map coordinate system as the global coordinate system;
  • Step S32 Solve the rotation and translation of the sparse point cloud of the first map relative to the sparse point cloud of the second map by using an iterative Closest Point (ICP) algorithm;
  • ICP iterative Closest Point
  • Step S33 updating all the sparse point clouds in the first map to the global coordinate system, and adding the sparse point cloud set of the first map to the sparse point cloud set of the second map;
  • Step S34 Repeat steps S32 and S33 to merge other local maps into the global coordinate system
  • Step S35 Use an optimization method to optimize the posture of the local map after fusion, eliminate the accumulated error of multiple local map fusion, and further improve the accuracy of map update.
  • step S32 the rotation and translation of the sparse point cloud of the first map relative to the sparse point cloud of the second map are solved by the ICP algorithm, and the following explanation is given here.
  • the ICP algorithm is essentially an optimal registration method based on the least squares method. The algorithm repeatedly selects the corresponding point pairs and calculates the optimal rigid body transformation until the convergence accuracy requirements of the correct registration are met.
  • the basic principle of the ICP algorithm is: in the target point cloud P and the source point cloud Q to be matched, find the nearest point (p l , q l ) according to certain constraints, and then calculate the optimal rotation R Translate T to minimize the error function, and the error function E(R,T) is Where h is the number of adjacent point pairs, p l is a point in the target point cloud P, q l is the closest point in the source point cloud Q corresponding to p l , R is the rotation matrix, and T is the translation vector.
  • step S31 to step S35 the purpose of fusing multiple predefined sparse point cloud maps can be achieved through visual features, and the predetermined sparse point cloud map does not need to store additional feature point descriptor information, which reduces the size of the offline map.
  • the embodiment of the present application provides a map fusion device, which includes each module included and each unit included in each module, which can be implemented by a processor in an electronic device; of course, it can also be implemented by Specific logic circuit implementation; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.
  • the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.
  • FIG. 5A is a schematic diagram of the composition structure of a map fusion device according to an embodiment of the application.
  • the device 500 includes a map acquisition module 501, a coordinate conversion module 502, a map fusion module 503, and a map optimization module 504, wherein: map acquisition
  • the module 501 is configured to obtain a first map set and a second map.
  • the first map set includes one or more different first maps, and the coordinate system of each of the first maps is the same as that of the second map.
  • the coordinate system is different; the coordinate conversion module 502 is configured to use the coordinate system of the second map as the global coordinate system, and convert the local coordinates of the first sampling point in each of the first maps into the global coordinate system, Obtain the first global coordinates of the first sampling point; a map fusion module 503 configured to fuse the first global coordinates of the first sampling point into the second map to obtain an initial global map; a map optimization module 504 , Configured to optimize the first global coordinates of the first sampling point in the initial global map to reduce the fusion error and obtain the target global map.
  • the coordinate conversion module 502 is configured to: take the coordinate system of the second map as the global coordinate system, and sequentially convert the local samples of the first sampling points in the first map that meet the conditions in the first map set. The coordinates are converted into the first global coordinates; the map fusion module 503 is configured to sequentially merge the first global coordinates of the first sampling points in the first map that meet the conditions in the first map set to the second map , Get the initial global map.
  • the coordinate conversion module 502 includes: a determination sub-module and a coordinate conversion sub-module; wherein the determination sub-module is configured to: match the second map in the first map set The first map is determined to be the target local map; the coordinate conversion sub-module is configured to: convert the local coordinates of the first sampling point in the target local map to the first global coordinates; the map fusion module 503 is configured to: The first local coordinates of the first sampling point in the target local map are merged into the second map to obtain the merged second map; from the remaining first maps in the first map set, determine the The new target local map matched with the fused second map to merge the first global coordinates of the first sampling point in the new target local map into the fused second map until every Until the first map is merged into the second map, the initial global map is obtained.
  • the determining sub-module includes: a feature matching unit configured to compare the image feature of the first sampling point in each of the first map with the image feature of the second sampling point in the second map. Image features are matched to obtain a first matching pair set corresponding to the first map; the first determining unit is configured to determine the first matching pair set satisfying the first condition as the first target matching pair set; The first map corresponding to the first target matching pair set is determined as the target local map.
  • the feature matching unit is configured to determine the similarity between the image feature of the i-th first sampling point in the first map and each of the second sampling points to obtain the similarity Set, i is an integer greater than 0; determine the second sampling point whose similarity meets the second condition in the similarity set as the target point; associate each target point with the corresponding first sampling point, Obtain a first set of matching pairs corresponding to the first map.
  • the coordinate conversion sub-module includes: a second determining unit configured to: determine from the second sampling points in the second map, the coordinates corresponding to the first sampling point in the target local map The target point matching the image feature; the first coordinate of the target local map relative to the second map is determined according to the local coordinates of each first sampling point in the target local map and the corresponding global coordinates of the target point Conversion relationship; a coordinate conversion unit configured to: according to the first coordinate conversion relationship, convert the local coordinates of the first sampling point in the target local map to the first global coordinates.
  • the second determining unit is configured to determine that each target point is located at the target point according to the local coordinates of each first sampling point in the target local map and the global coordinates of the corresponding target point.
  • the local coordinates in the local map of the target; and the first coordinate conversion relationship is determined according to the global coordinates and local coordinates of each target point.
  • the determining sub-module includes: an iterative unit and a third determining unit; wherein the iterative unit is configured to, according to an iteration strategy, set the nth first map in the first map set The local coordinates of each first sampling point are matched with the global coordinates of multiple second sampling points in the second map to obtain a matching result, where n is an integer greater than 0; the third determining unit is configured to When the matching result indicates that the matching is successful, the n-th first map is determined as the target local map; the iteration unit is further configured to continue to perform the next step when the matching result indicates that the matching fails.
  • the local coordinates of each first sampling point in a first map are matched with the global coordinates of the multiple second sampling points until the target local map is determined from the first map set.
  • the iterative unit includes: a selection subunit configured to select an initial sampling point that matches each first sampling point in the nth first map from the plurality of second sampling points Target point; a determining subunit, configured to determine the nth first map relative to the nth first map based on the local coordinates of each first sampling point in the nth first map and the corresponding global coordinates of the initial target point The second coordinate conversion relationship of the second map; according to the second coordinate conversion relationship, the local coordinates of each first sampling point in the nth first map, and the global coordinates of the corresponding initial target point, the matching is determined Error; if the matching error is greater than the first threshold, reselect the initial target point, and re-determine the matching error; the result generation unit is configured to generate a match that characterizes successful matching if the matching error is less than or equal to the first threshold result.
  • the iterative unit is further configured to: if it is determined that the number of matching errors is greater than the second threshold, generate a matching result that characterizes the failure of the matching, and continue to select and download from the plurality of second sampling points. An initial target point matched by each first sampling point in a first map until a matching result characterizing a successful match is generated.
  • the coordinate conversion unit is configured to determine the second coordinate conversion relationship when the determined matching error is less than or equal to the first threshold when the matching result indicates that the matching is successful.
  • the selection subunit is configured to: obtain a third coordinate conversion relationship of the nth first map with respect to the second map; according to the third coordinate conversion relationship and the first The local coordinates of the j-th first sampling point in the n first maps determine the second global coordinate of the j-th first sampling point, where j is an integer greater than 0; compare the second global coordinates with the The global coordinates of a plurality of second sampling points are matched to obtain an initial target point matching the j-th first sampling point.
  • the determining subunit is configured to determine the j-th first sampling point according to the second coordinate conversion relationship and the local coordinates of the j-th first sampling point in the n-th first map.
  • the third global coordinate of the sampling point, j is an integer greater than 0; the matching is determined according to the third global coordinate of each first sampling point in the nth first map and the global coordinate of the corresponding initial target point error.
  • the determining subunit is configured to determine the distance between the third global coordinate of each first sampling point in the nth first map and the global coordinate of the corresponding initial target point; According to each of the distances, the matching error is determined.
  • the determining subunit is configured to, if the matching error is greater than the first threshold, use the second coordinate conversion relationship as the third coordinate conversion relationship, and reselect an initial target point.
  • the map optimization module 504 includes: an acquisition sub-module configured to acquire the first target matching pair set determined when each of the first maps is merged into the current second map,
  • the first target matching pair set includes the i-th first sampling point in the first map and the second sampling point that matches the i-th first sampling point in the current second map, where i is An integer greater than 0;
  • the coordinate optimization sub-module is configured to optimize the first global coordinate of each first sampling point in the set for each of the first target matching to obtain the target global coordinate of the corresponding first sampling point
  • the update submodule is configured to update the target global coordinates of each first sampling point to the initial global map to obtain the target global map.
  • the coordinate optimization sub-module includes: a third determining unit configured to determine the corresponding first global coordinate of each first sampling point in each first target matching pair set The reprojection error of the first sampling point; the coordinate adjustment unit is configured to adjust the first global coordinate of each first sampling point in the set of each of the first target matching pairs, so that the reprojection error of each first sampling point The third condition is satisfied, and the target global coordinate of the corresponding first sampling point is obtained.
  • the third determining unit is configured to: obtain a projection matrix corresponding to the nth first map in the first map set; obtain a first coordinate conversion relationship, where the first coordinate conversion relationship is The coordinate conversion relationship of the nth first map relative to the current second map when being fused, n is an integer greater than 0; the depth information of the kth first sampling point is acquired, and the kth first sampling Point is the first sampling point in the first target matching pair set corresponding to the nth first map; according to the projection matrix, the first coordinate conversion relationship, and the kth first sampling point A global coordinate and the depth information determine the reprojection error of the k-th first sampling point, where k is an integer greater than 0.
  • the third determining unit is configured to: according to the projection matrix, the first coordinate conversion relationship, the first global coordinates of the kth first sampling point, and the depth information, Determine the projection coordinates of the k-th first sampling point in the current second map; determine the projection coordinates of the k-th first sampling point according to the projection coordinates and the pixel coordinates of the target point that matches the k-th first sampling point The reprojection error of the k first sampling points.
  • the coordinate adjustment unit is configured to: determine a reprojection error function value according to the reprojection error of each first sampling point in the set of each of the first target matching pairs; The first global coordinates of the first sampling point corresponding to each of the reprojection errors are used to minimize the value of the reprojection error function to obtain the value of each first sampling point in the set of each of the first target matching pairs The global coordinates of the target.
  • the coordinate adjustment unit is configured to: adjust each of the first coordinate conversion relationships to obtain a corresponding fourth coordinate conversion relationship; According to the fourth coordinate conversion relationship, the first global coordinate of the k-th first sampling point is re-determined; according to the depth information of the k-th first sampling point and the re-determined first global coordinate, the fourth The coordinate conversion relationship and the projection matrix are used to re-determine the re-projection error of the k-th first sampling point; the re-projection error of each first sampling point in the set is re-determined according to each of the first target matches , Re-determine the re-projection error function value; if the re-determined re-projection error function value is less than the second threshold, determine the fourth global coordinate as the target global coordinate.
  • the coordinate adjustment unit is further configured to: if the re-determined re-projection error function value is greater than or equal to the second threshold, continue to adjust the fourth coordinate conversion relationship to re-determine the re-projection error Function value, until the currently determined reprojection error function value is less than the second threshold, the currently determined first global coordinate is determined as the target global coordinate.
  • the device 500 further includes: a determining module 505 configured to determine at least one first sample point according to the local coordinates and image features of the first sample point in the multiple sample images The global coordinates of this point; the first data set is determined according to the global coordinates and image features of each of the first sample points; the local coordinates and image features of the second sample points in the acquired m-th other sample images, Determine a second data set corresponding to the m-th other sample image, where m is an integer greater than 0; determine the global coordinates of the second sample point in the second data set according to the first data set; map building module And configured to construct the second map at least according to the global coordinates of each of the first sample points and the global coordinates of each of the second sample points.
  • a determining module 505 configured to determine at least one first sample point according to the local coordinates and image features of the first sample point in the multiple sample images The global coordinates of this point; the first data set is determined according to the global coordinates and image features of each of the first sample points; the local
  • the determining module 505 is configured to select a first target satisfying the fourth condition from the multiple sample images according to the local coordinates and image characteristics of each of the first sample points Image and a second target image; determine a fifth coordinate conversion relationship between the first target image and the second target image; according to the fifth coordinate conversion relationship and the first target image The local coordinates of this point determine the global coordinates of the first sample point in the first target image.
  • the determining module 505 is configured to: perform pairwise matching of the multiple sample images according to the image characteristics of each of the first sample points to obtain the second sample image of each pair of sample images. Matching pair set; removing matching pairs that do not meet the fourth condition in the first matching pair set to obtain a third matching pair set; from each third matching pair set, selecting the number of matching pairs that satisfies the first set of matching pairs A four-condition second target matching pair set; the two sample images corresponding to the second target matching pair set are determined as the first target image and the second target image.
  • the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to enable An electronic device (which may be a mobile phone, a tablet computer, a notebook computer, a desktop computer, a robot, a drone, a server, etc.) executes all or part of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), magnetic disk or optical disk and other media that can store program codes. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.
  • FIG. 6 is a schematic diagram of a hardware entity of the electronic device according to an embodiment of the application.
  • the hardware entity of the electronic device 600 includes: a memory 601 and a processor. 602.
  • the memory 601 stores a computer program that can run on the processor 602, and the processor 602 implements the steps in the map fusion method provided in the foregoing embodiment when the processor 602 executes the program.
  • the memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and each module in the electronic device 600 (for example, image data, audio data, voice communication data, and Video communication data) can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).
  • an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the map fusion method provided in the foregoing embodiments are implemented.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, such as: multiple units or components can be combined, or It can be integrated into another system, or some features can be ignored or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms. of.
  • the units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the functional units in the embodiments of the present application can be all integrated into one processing unit, or each unit can be individually used as a unit, or two or more units can be integrated into one unit;
  • the unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.
  • the foregoing program can be stored in a computer readable storage medium.
  • the execution includes The steps of the foregoing method embodiment; and the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a read only memory (Read Only Memory, ROM), a magnetic disk, or an optical disk.
  • ROM Read Only Memory
  • the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer readable storage medium.
  • the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to enable An electronic device (which may be a mobile phone, a tablet computer, a notebook computer, a desktop computer, a robot, a drone, a server, etc.) executes all or part of the method described in each embodiment of the present application.
  • the aforementioned storage media include: removable storage devices, ROMs, magnetic disks or optical discs and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例公开了地图融合方法及装置、设备、存储介质，其中，所述方法包括：获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。

Description

地图融合方法及装置、设备、存储介质
相关申请的交叉引用
本申请基于申请号为201910923582.3、申请日为2019年09月27日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以全文引入的方式引入本申请。
技术领域
本申请实施例涉及电子技术，涉及但不限于地图融合方法及装置、设备、存储介质。
背景技术
目前，通过视觉信息可以建立室内环境地图，在构建室内环境地图的过程中不可避免地遇到地图融合的需求。比如，在多次数据采集后的地图构建、多人或多机器协作完成地图构建等应用场景中，都需要将在各个局部区域对应的局部地图合成为一个全局地图。然而，合成的全局地图却存在较大的融合误差，在合成后多个局部地图之间的一致性较低。
发明内容
有鉴于此，本申请实施例提供的地图融合方法及装置、设备、存储介质，能够减小地图融合误差，改善多个局部地图融合后得到的全局地图的一致性问题。本申请实施例的技术方案是这样实现的：
本申请实施例提供的地图融合方法，包括：获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，得到目标全局地图。
本申请实施例提供的地图融合装置，包括：地图获取模块，配置为获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；坐标转换模块，配置为以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；地图融合模块，配置为将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；地图优化模块，配置为对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，得到目标全局地图。
本申请实施例提供的电子设备，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，处理器执行所述程序时实现上述地图融合方法中的步骤。
本申请实施例提供的计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述地图融合方法中的步骤。
在本申请实施例中，将一个或多个不同的第一地图中第一采样点的第一全局坐标融合到第二地图中，得到初始全局地图之后，对初始全局地图中第一采样点的第一全局坐标进行优化，得到目标全局地图；如此，可以减小地图融合误差，改善该初始全局地图的一致性问题。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本申请的实施例，并于说明书一起用于说明本申请的技术方案。
图1A为本申请实施例地图融合方法的实现流程示意图；
图1B为本申请实施例将多个第一地图融合到第二地图的示意图；
图2为本申请实施例确定多个目标点的局部坐标的示意图；
图3为本申请实施例特征点匹配对示意图；
图4为本申请实施例重投影误差的示意图；
图5A为本申请实施例地图融合装置的组成结构示意图；
图5B为本申请实施例另一地图融合装置的组成结构示意图；
图6为本申请实施例电子设备的一种硬件实体示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请的具体技术方案做进一步详细描述。以下实施例用于说明本申请，但不用来限制本申请的范围。除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本申请实施例的目的，不是旨在限制本申请。在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。需要指出，本申请实施例所涉及的术语“第一\第二\第三”仅仅是是区别不同的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使这里描述的本申请实施例能够以除了在这里图示或描述的以外的顺序实施。
本申请实施例提供一种地图融合方法，所述方法可以应用于电子设备，所述电子设备可以是手机、平板电脑、笔记本电脑、台式计算机、机器人、无人机或服务器等具有信息处理能力的设备。所述地图融合方法所实现的功能可以通过所述电子设备中的处理器调用程序代码来实现，当然程序代码可以保存在计算机存储介质中。可见，所述电子设备至少包括处理器和存储介质。
图1A为本申请实施例地图融合方法的实现流程示意图，如图1A所示，所述方法至少可以包括以下步骤S101至步骤S104：
步骤S101，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同。
需要说明的是，多个不同的第一地图可以是多个不同的电子设备在不同区域通过内置的图像采集模组所采集的局部地图，或者还可以是一个电子设备在不同区域通过内置的图像采集模组所采集的局部地图。多个不同的电子设备实施采集时，这些电子设备可以以众包的形式将采集的第一地图发送给实施所述地图融合方法的电子设备。第二地图实际上也是一个局部地图，但是与每一第一地图相比，可能采集区域是不同的。
对于第二地图的坐标系的类型不做限定，该地图的坐标系可以为适合某种应用场景下的坐标系，例如在视觉定位中，第二地图的坐标系为世界坐标系。当然，该地图的坐标系还可以是自定义的坐标系。
步骤S102，以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标。
电子设备在实现步骤S102时，可以将第一地图中与第二地图不匹配的第一采样点的局部坐标转换为第一全局坐标；在执行步骤S103时，将这些不匹配的第一采样点的第一全局坐标融合到第二地图中。例如，将第一地图中采样点的图像特征与第二地图中的图像特征不匹配的采样点的第一全局坐标添加至第二地图中。也就是，将第一地图中与第二地图不相重叠的区域的采样点的第一全局坐标添加至第二地图中。
在一些实施例中，电子设备还可以将第一地图中的全部第一采样点的第一全局坐标融合到第二地图中。需要说明的是，每一第一采样点的第一全局坐标是不同的。第一采样点不是特指第一地图中的某一采样点，而是为了与第二采样点区别开来。也就是，第一采样点泛指第一地图中的采样点。
步骤S103，将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图。
电子设备在实现步骤S103时，可以将多个第一采样点的第一全局坐标添加到第二地图中。在一些实施例中，电子设备还可以将多个第一采样点的图像特征和第一全局坐标添加到第二地图中。
步骤S104，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，得到目标全局地图。
可以理解地，优化的目的是为了得到与第一采样点的实际坐标更为接近的目标全局坐标，以减 小在确定第一采样点的第一全局坐标时所引入的累计误差，从而减小融合误差，得到目标全局地图。
可以理解地，所谓局部地图和全局地图，“局部”和“全局”是相对而言的，前者是指特定物理范围内的部分区域，而后者则是指该物理范围的整个区域。以该特定物理范围为一栋写字楼为例，局部地图是指某一楼层或者某一楼层的某一房间对应的地图，而全局地图则是指该写字楼对应的地图。
在本申请实施例中，电子设备在将每一第一地图中第一采样点的第一全局坐标融合到第二地图之后，不是直接将得到的初始全局地图作为最终应用的目标全局地图，而是，对初始全局地图中第一采样点的第一全局坐标进行优化；如此可以减小融合误差，使得到的第一采样点的第一全局坐标更加接近实际坐标。
这是因为，在将多个第一地图融合到第二地图中，通常当前第一地图中第一采样点的第一全局坐标是在前一个第一地图被融合到第二地图的基础上所确定的，这样就带来了累计误差，即，随着融合到第二地图的第一地图数量的增加，得到的第一采样点的第一全局坐标的误差越随之增加。因此，在得到初始全局地图之后，还需要对融合进来的第一采样点的第一全局坐标进行优化，以减小累计误差。
本申请实施例再提供一种地图融合方法，所述方法可以包括以下步骤S201至步骤S204：
步骤S201，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S202，以所述第二地图的坐标系为全局坐标系，依次将所述第一地图集合中满足条件的第一地图中第一采样点的局部坐标，转换为第一全局坐标；
步骤S203，依次将所述第一地图集合中满足条件的第一地图中第一采样点的第一全局坐标，融合到当前第二地图中，得到所述初始全局地图。
电子设备可以通过如下实施例中的步骤S302至步骤S305实现步骤S202和步骤S203，即，优先将第一地图集合中与第二地图相匹配的第一地图中第一采样点的局部坐标，转换为第一全局坐标，并融合到当前第二地图中。例如，优先将与第二地图之间具有匹配对数目最多的第一地图融合至当前第二地图中，也就是将与第二地图重叠区域最多的第一地图率先融合至当前第二地图中。
步骤S204，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，得到目标全局地图。
在本申请实施例中，优先融合满足条件的第一地图，这样不仅可以提高融合精度，还可以避免出现融合失败的情况，从而提高地图融合效率。
本申请实施例再提供一种地图融合方法，所述方法至少包括以下步骤S301至步骤S306：
步骤S301，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S302，将所述第一地图集合中与所述第二地图相匹配的第一地图，确定为目标局部地图。
电子设备可以通过如下实施例中的步骤S402至步骤S404实现步骤S302；或者，所述电子设备还可以通过如下实施例中的步骤S502至步骤S504实现步骤S302。
步骤S303，以所述第二地图的坐标系为全局坐标系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
电子设备可以通过如下实施例中的步骤S311至步骤S313、或者步骤S405、或者步骤S502至步骤S504来实现步骤S303。
步骤S304，将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；
步骤S305，从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图。
举例来说，如图1B所示，假设第一地图集合10包括的第一地图为地图11、地图12和地图13，在进行第一次地图融合时，第一地图集合10中与第二地图14(即当前第二地图)相匹配的第一地图为地图12，此时将地图12作为目标局部地图，融合到第二地图14中，得到的融合后的第二地图14为地图141；在进行第二次地图融合时，从第一地图集合中确定与地图141(即当前第二地图)相匹配的新的目标局部地图为地图11，此时将地图11融合到地图141中，得到的融合后的第二地图为地图142；最后，将地图13融合到地图142(即当前第二地图)中，得到初始全局地图143。
步骤S306，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
在本申请实施例中，确定待融合的第一地图(即目标局部地图)和地图融合，这两个动作交替进行，即新的目标局部地图是在当前得到的第二地图的基础上所确定的；这样，使得电子设备能够从第一地图集合中找到与当前得到的第二地图具有更多重叠区域的目标局部地图，这对于目标局部地图中第一采样点的第一全局坐标的确定精度是有利的，能够减小融合误差，提高地图融合精度。
在一些实施例中，电子设备可以通过如下步骤S311至步骤S313实现上述步骤S303(即所述以所述第二地图的坐标系为全局坐标系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标的步骤)：
步骤S311，从所述第二地图的第二采样点中，确定出与所述目标局部地图中第一采样点的图像特征相匹配的目标点；
步骤S312，根据所述目标局部地图中多个第一采样点的局部坐标和与所述多个第一采样点分别对应的目标点的全局坐标，确定所述目标局部地图相对于所述第二地图的第一坐标转换关系。
在实现步骤S312时，电子设备可以根据所述多个第一采样点的局部坐标以及与所述多个第一采样点分别对应的目标点的全局坐标，确定与所述多个第一采样点分别对应的目标点在所述目标局部地图中的局部坐标；根据每一目标点的全局坐标和局部坐标，确定所述第一坐标转换关系。
在一些实施例中，第一坐标转换关系包括目标局部地图相对于第二地图的旋转关系，电子设备可以根据所述旋转关系，确定图像采集模组的镜头在采集所述目标局部地图时相对于图像采集模组的镜头在采集第二地图时的朝向。
一般来说，电子设备可以根据至少3个第一采样点的局部坐标以及与所述3个第一采样点分别匹配的目标点的全局坐标，准确地确定出与所述3个第一采样点分别匹配的目标点的局部坐标。
举例来说，如图2所示，点O为目标局部地图的坐标系的原点，多个目标点为图2中所示的大写的A、B、C这3个点，在目标局部地图中，与点A匹配的第一采样点为小写的点a，与点B匹配的第一采样点为小写的点b，与点C匹配的第一采样点为小写的点c。
根据余弦定理可以列出如下公式(1)：
OA 2+OB 2-2·OA·OB·cos＜a,b＞＝AB 2
OA 2+OC 2-2·OA·OC·cos＜a,c＞＝AC 2     (1)；
OB 2+OC 2-2·OB·OC·cos＜b,c＞＝BC 2
式(1)中，＜a,b＞指的是∠aOb，＜a,c＞指的是∠aOc，＜b,c＞指的是∠bOc。OA、OB和OC分别为目标局部地图的坐标系的原点与目标点A、B、C的距离。
对上式进行消元，同时除以OC 2，并令
Figure PCTCN2020116926-appb-000001
则可得出如下式(2)：
Figure PCTCN2020116926-appb-000002
接着进行替换，令
Figure PCTCN2020116926-appb-000003
则可得出如下式(3)：
x 2+y 2-2·x·y·cos＜a,b＞＝u 2
x 2+1-2·x·cos＜a,c＞＝wu    (3)；
y 2+1-2·y·cos＜b,c＞＝vu
将上面的式(1)带入式(2)和式(3)，则可得出如下式(4)：
Figure PCTCN2020116926-appb-000004
在式(4)中，w、v、cos＜a,c＞、cos＜b,c＞、cos＜a,b＞都是已知量，因此未知量有x和y两个，因此通过上式(4)中的两个方程可以求得x和y的值，接着根据如下公式(5)的三个方程就可以求解OA、OB和OC的值：
Figure PCTCN2020116926-appb-000005
最后求解A、B、C这3个目标点的局部坐标，根据向量公式(6)，可得：
Figure PCTCN2020116926-appb-000006
式(6)中，
Figure PCTCN2020116926-appb-000007
的方向为点O到点a；
Figure PCTCN2020116926-appb-000008
的方向为点O到点b；
Figure PCTCN2020116926-appb-000009
的方向为点O到点c。
步骤S313，根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
电子设备在实现步骤S313时，可以将目标局部地图中的每一第一采样点的局部坐标转换为第一全局坐标；还可以将目标局部地图中的部分第一采样点的局部坐标转换为第一全局坐标，例如，将目标局部地图中的除与目标点匹配的第一采样点外的其他第一采样点的局部坐标转换为第一全局坐标。
本申请实施例再提供一种地图融合方法，所述方法可以包括以下步骤S401至步骤S410：
步骤S401，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S402，将每一所述第一地图中第一采样点的图像特征，分别与所述第二地图中第二采样点的图像特征进行匹配，得到与所述第一地图对应的第一匹配对集合。
通过步骤S402，电子设备可以得到与每一第一地图对应的第一匹配对集合。换句话说，每一第一匹配对集合都可以通过步骤S402来获得。例如，电子设备可以通过以下步骤来确定与所述第一地图对应的第一匹配对集合，即：确定所述第一地图中第i个第一采样点的图像特征与每一所述第二采样点之间的相似度，得到相似度集合，i为大于0的整数；将所述相似度集合中相似度满足第二条件的第二采样点，确定为目标点；将每一所述目标点和对应的第一采样点进行关联，得到与所述第一地图对应的第一匹配对集合。
可以理解地，第一匹配对集合中包括的匹配对为第一采样点和与其匹配的第二采样点(即目标点)。所述相似度，指的是两个采样点的图像特征之间的相近程度。在实现时，所述相似度可以通过欧氏距离、汉明距离或者余弦相似度等来表征。第二条件可以是多种多样的，在本申请中对此不做限定。例如，第二条件为相似度小于第三阈值，又如，第二条件为相似度集合中相似度最小。
步骤S403，将满足第一条件的第一匹配对集合，确定为第一目标匹配对集合。
同样地，第一条件也可以是多种多样的。例如，第一条件为匹配对数目大于第四阈值，又如，第一条件为匹配对数目最多。
步骤S404，将与所述第一目标匹配对集合对应的第一地图，确定为目标局部地图；
步骤S405，根据所述第一目标匹配对集合中多个第一采样点的局部坐标和对应的目标点的全局坐标，确定所述目标局部地图相对于所述第二地图的第一坐标转换关系；
步骤S407，以所述第二地图的坐标系为全局坐标系，根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标；
步骤S408，将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；
步骤S409，从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图；
步骤S410，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
在本申请实施例中，基于第一采样点的图像特征和第二采样点的图像特征，从第一地图集合中选出与第二地图相匹配的目标局部地图；这样，使得所述电子设备能够找到与第二地图更加匹配的 目标局部地图，这是因为根据图像特征，能够获得更好的匹配准确度；如此，也更加利于提高融合精度。
本申请实施例再提供一种地图融合方法，所述方法可以包括以下步骤S501至步骤S508：
步骤S501，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S502，根据迭代策略，将所述第一地图集合中第n个第一地图的每一第一采样点的局部坐标与所述第二地图中多个第二采样点的全局坐标进行匹配，得到匹配结果，n为大于0的整数。
可以理解地，每一第一地图包括第一采样点的局部坐标，第二地图包括第二采样点的全局坐标，但是这些地图均不包括采样点的图像特征；如此，可以使得地图融合方法适用于所有局部地图均没有图像特征的应用场景中，即，在第二地图和每一第一地图没有采样点的图像特征的前提下，电子设备也能够实现地图融合。另外，电子设备在存储目标全局地图时，图像特征一般占据比较大的存储空间。例如，图像特征为特征描述子，通常情况下，每个采样点的特征描述子具有256个字节，这就需要电子设备给每个采样点分配至少256个字节的存储空间来存储特征描述子。在实现时，目标全局地图中不包括采样点的图像特征；如此，可以大大降低目标全局地图的数据量，从而节约电子设备存储目标全局地图消耗的存储资源。
步骤S503，在所述匹配结果表征匹配成功的情况下，将所述第n个第一地图确定为所述目标局部地图；
步骤S504，在所述匹配结果表征匹配失败的情况下，继续将下一个第一地图中每一第一采样点的局部坐标与所述多个第二采样点的全局坐标进行匹配，直到从所述第一地图集合中确定出所述目标局部地图为止，执行步骤S505；
步骤S505，以所述第二地图的坐标系为全局坐标系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标；
步骤S506，将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；
步骤S507，从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图；
步骤S508，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
本申请实施例再提供一种地图融合方法，所述方法可以包括以下步骤S601至步骤S614：
步骤S601，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S602，从所述第二地图的多个第二采样点中，选取与所述第n个第一地图中每一第一采样点匹配的初始目标点。
电子设备在实现步骤S602时，可以设置第一地图相对于第二地图的初始坐标转换关系；然后，根据第一地图中第一采样点的局部坐标和所述初始转换关系，将第一地图中的第一采样点与所述多个第二采样点进行匹配，从而从所述多个第二采样点中选取与所述第一地图中的第一采样点匹配的初始目标点。在一些实施例中，电子设备可以通过如下实施例中的步骤S702至步骤S704，选取所述初始目标点。
实际上，通过步骤S602，是为了选取与第一地图中的第一采样点可能匹配的第二采样点，初始目标点可能不是与第一采样点真正匹配的点，因此，需要通过如下步骤S603至步骤S610，进一步确定初始目标点是否是与第一采样点真正匹配的点。
步骤S603，根据所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定所述第n个第一地图相对于所述第二地图的第二坐标转换关系。
电子设备在实现步骤S603时，可以根据第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，构建误差函数；然后，通过最小二乘法求解当前最优的第二坐标转换关系。例如，包括h个第一采样点的局部坐标的集合表示为P＝{p 1,p 2,...,p l,...,p h}，第一采样点的局部坐标用p l来表示，与所述h个第一采样点匹配的初始目标点的全局坐标的集合表示为Q＝{q 1,q 2,...,q l,...,q h}，初始目标点的全局坐标用q l来表示，那么，可以列出如下式(7)：
Figure PCTCN2020116926-appb-000010
式中，E(R,T)为误差函数，R和T分别为待求解的第二坐标转换关系中的第二旋转关系和第二平移关系。那么，可以通过最小二乘法求解式(7)中R和T的最优解。
步骤S604，根据所述第二坐标转换关系、所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定匹配误差。
电子设备在实现步骤S604时，可以通过如下实施例中的步骤S706和步骤S707确定所述匹配误差。
步骤S605，统计确定匹配误差的次数。
可以理解地，在处理当前的第一地图时，每确定一次匹配误差，就统计确定匹配误差的次数，在处理下一个第一地图时，可以将当前次数清零。
步骤S606，确定所述次数是否大于第二阈值；如果是，执行步骤S607；否则，执行步骤S608。
可以理解地，如果所述次数大于第二阈值，说明第n个第一地图与当前第二地图是不匹配的，例如两者没有相匹配的采样点。此时，可以将当前统计的确定匹配误差的次数清零之后，将第n个第一地图替换为第n+1个(即下一个)第一地图，重新执行类似步骤S602至步骤606的内容。
步骤S607，生成表征匹配失败的匹配结果，并继续从所述多个第二采样点中，选取与下一个第一地图中每一第一采样点匹配的初始目标点，直到生成表征匹配成功的匹配结果为止，进入步骤S610。
步骤S608，确定所述匹配误差是否大于第一阈值；如果是，返回步骤S602，重新选取初始目标点，并重新确定匹配误差；否则，执行步骤S609。
步骤S609，生成表征匹配成功的匹配结果。
步骤S610，在所述匹配结果表征匹配成功的情况下，将所述第n个第一地图确定为所述目标局部地图，并将确定的匹配误差小于或等于所述第一阈值时的第二坐标转换关系，确定为所述目标局部地图相对于所述第二地图的第一坐标转换关系。
可以理解地，如果所述匹配误差大于第一阈值，说明当前选取的初始目标点并不是与当前第一地图中第一采样点相匹配的点，两者指代的不是物理空间中同一位置点或者相近的位置点。此时，还需要返回步骤S602，重新选取初始目标点，并基于重新选取的初始目标点，重新执行步骤S603至步骤S604，以重新确定匹配误差，直至重新确定的匹配误差小于所述第一阈值时，认为当前迭代中选取的初始目标点是与当前第一地图中第一采样点真正匹配的点，此时可以将当前迭代获得的第二坐标转换关系确定为第一坐标转换关系
步骤S611，以所述第二地图的坐标系为全局坐标系，根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
步骤S612，将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图。
步骤S613，从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图。
步骤S614，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
本申请实施例再提供一种地图融合方法，所述方法可以包括以下步骤S701至步骤S717：
步骤S701，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S702，获取所述第n个第一地图相对于所述第二地图的第三坐标转换关系；在实现时，可以将所述第三坐标转换关系设置一个初始值；
步骤S703，根据所述第三坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第二全局坐标，j为大于0的整数；
步骤S704，将所述第二全局坐标与所述多个第二采样点的全局坐标进行匹配，得出与所述第j个第一采样点匹配的初始目标点。
电子设备在实现步骤S704时，可以确定所述第j个第一采样点的第二全局坐标与每一所述第二采样点的全局坐标之间的距离(例如欧式距离)；然后将距离所述第j个第一采样点最近的第二采样 点确定为初始目标点，或者将距离小于或等于距离阈值的第二采样点确定为初始目标点。
步骤S705，根据所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定所述第n个第一地图相对于所述第二地图的第二坐标转换关系；
步骤S706，根据第二坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第三全局坐标，j为大于0的整数；
步骤S707，根据所述第n个第一地图中每一第一采样点的第三全局坐标和对应的初始目标点的全局坐标，确定所述匹配误差。
电子设备在实现步骤S707时，可以先确定所述第n个第一地图中每一第一采样点的第三全局坐标与对应的初始目标点的全局坐标之间的距离(例如欧式距离等)；根据每一所述距离，确定所述匹配误差。
在一些实施例中，电子设备可以将多个第一采样点与匹配的初始目标点之间的平均距离，确定为所述匹配误差。例如，包括h个第一采样点的第三全局坐标p′ l的集合表示为P′＝{p′ 1,p′ 2,...,p′ l,...,p′ h}，与所述h个第一采样点匹配的初始目标点的全局坐标q的集合表示为Q＝{q 1,q 2,...,q l,...,q h}，那么通过如下公式(8)可以求取匹配误差d：
Figure PCTCN2020116926-appb-000011
式中||p′ l-q l|| 2表示第一采样点与匹配的初始目标点之间的欧式距离。
步骤S708，统计确定匹配误差的次数；
步骤S709，确定所述次数是否大于第二阈值；如果是，执行步骤S710；否则，执行步骤S711；
步骤S710，生成表征匹配失败的匹配结果，并返回执行步骤S702，继续获取下一个第一地图相对于所述第二地图的第三坐标转换关系，直到生成表征匹配成功的匹配结果为止，进入步骤S713；
步骤S711，确定所述匹配误差是否大于第一阈值；如果是，将所述第二坐标转换关系作为所述第三坐标转换关系，然后返回执行步骤S703，以重新选取初始目标点；否则，执行步骤S712。
可以理解地，如果所述匹配误差大于第一阈值，说明获取的第三坐标转换关系是不符合实际的。换句话说，得出的初始目标点不是真正与第一采样点相匹配的点，此时，可以将第二坐标转换关系作为所述第三坐标转换关系，重新执行步骤S703至步骤S710，直至匹配误差小于所述第一阈值为止，执行步骤S712。
步骤S712，生成表征匹配成功的匹配结果；
步骤S713，在所述匹配结果表征匹配成功的情况下，将所述第n个第一地图确定为所述目标局部地图。
需要说明的是，如果表征匹配成功的匹配结果为所述电子设备将下一个第一地图中每一第一采样点的局部坐标与所述多个第二采样点的全局坐标进行匹配所得到的匹配结果，则此时将所述下一个第一地图确定为目标局部地图。也就是说，在当前匹配结果表征匹配成功的情况下，将当前被匹配的第一地图确定为目标局部地图。
步骤S714，以所述第二地图的坐标系为全局坐标系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标；
步骤S715，将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；
步骤S716，从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图；
步骤S717，对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
本申请实施例再一种地图融合方法，所述方法可以包括以下步骤S801至步骤S806：
步骤S801，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S802，以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；
步骤S803，将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；
步骤S804，获取每一所述第一地图在被融合到当前第二地图中时，所确定的第一目标匹配对集合，所述第一目标匹配对集合包括所述第一地图中第i个第一采样点、和所述当前第二地图中与所述第i个第一采样点匹配的第二采样点，i为大于0的整数。
可以理解地，每一第一地图都是在前一第一地图被融合到第二地图的基础上进行融合的，也就是说，可以将前一次融合后的第二地图理解为当前第二地图。例如，以图1B所示的地图融合方式为例，地图12被融合到的当前第二地图为第二地图14，融合后的第二地图14为地图141，地图11被融合到的当前第二地图为地图141。
由前述实施例可知，电子设备在获取第一目标匹配对集合之前，可以通过类似于上述步骤S402和步骤S403来确定第一目标匹配对集合，这里，直接获取即可。
步骤S805，对每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标进行优化，得到对应的第一采样点的目标全局坐标；
步骤S806，将每一第一采样点的目标全局坐标更新至所述初始全局地图中，得到所述目标全局地图。
在本申请实施例中，对第一目标匹配对集合中具有第一全局坐标的第一采样点进行坐标优化，这样，可以降低执行步骤S805的计算量，提高获得目标全局地图的效率。
在一些实施例中，对于上述步骤S805至少可以通过如下步骤S811至步骤S812来实现：
步骤S811，根据每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标，确定对应的第一采样点的重投影误差。
以所述第一目标匹配对集合对应的第一地图为第n个第一地图为例，确定所述第一目标匹配对集合中第一采样点的重投影误差可以通过如下步骤实现：获取所述第一地图集合中第n个第一地图对应的投影矩阵；获取第一坐标转换关系，所述第一坐标转换关系为所述第n个第一地图在被融合时相对于当前第二地图的坐标转换关系，n为大于0的整数；获取第k个第一采样点的深度信息，所述第k个第一采样点为所述第n个第一地图对应的第一目标匹配对集合中的第一采样点；根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点的重投影误差，k为大于0的整数。
步骤S812，调整每一所述第一目标匹配对集合中每一第一采样点的第一全局坐标，使得每一第一采样点的重投影误差满足第三条件，从而得到对应的第一采样点的目标全局坐标。
可以理解地，通过步骤S812可以同时调整每一第一采样点的第一全局坐标，从而使得每一第一采样点的重投影误差满足第三条件。其中，第三条件例如可以为由每一第一采样点的重投影误差确定的重投影误差函数值小于第二阈值，这样进而可以得到每一第一采样点的目标全局坐标。
电子设备可以这样实现步骤S812，即，根据每一所述第一目标匹配对集合中每一第一采样点的重投影误差，确定重投影误差函数值；迭代地调整在确定每一所述重投影误差时对应的第一采样点的第一全局坐标，以最小化所述重投影误差函数值，得到每一所述第一目标匹配对集合中每一第一采样点的目标全局坐标。
在一些实施例中，对于上述步骤S811，所述根据每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标，确定对应的第一采样点的重投影误差，至少可以通过以下步骤S821至步骤S824来实现：
步骤S821，获取所述第一地图集合中第n个第一地图对应的投影矩阵；
步骤S822，获取第一坐标转换关系，所述第一坐标转换关系为所述第n个第一地图在被融合时相对于当前第二地图的坐标转换关系，n为大于0的整数；
步骤S823，获取第k个第一采样点的深度信息，所述第k个第一采样点为所述第n个第一地图对应的第一目标匹配对集合中的第一采样点；
步骤S824，根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点的重投影误差，k为大于0的整数。
电子设备可以这样实现步骤S824：根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点在所述当前第二地图中的投影坐标；根据所述投影坐标和与所述第k个第一采样点匹配的目标点的像素坐标，确定所述第k个第一采样点的重投影误差。
假设所述第k个第一采样点的第一全局坐标为P k＝[X k,Y k,Z k] T，其在当前第二地图中的投影坐 标为U k＝[u k,v k] T，两者之间的关系如下公式(9)所示：
Figure PCTCN2020116926-appb-000012
式中，s k指的是所述第k个第一采样点的深度信息，K表示所述投影矩阵，exp(ξ)表示所述第一坐标转换关系。
由公式(9)可以得出投影坐标U k的计算公式为如下公式(10)：
Figure PCTCN2020116926-appb-000013
基于此，根据每一所述第一目标匹配对集合中每一第一采样点的重投影误差，确定重投影误差函数值ζ *的公式如下式(11)所示：
Figure PCTCN2020116926-appb-000014
式中，
Figure PCTCN2020116926-appb-000015
表示第k个第一采样点的重投影误差，μ k表示与所述第k个第一采样点匹配的目标点的像素坐标。
这样，电子设备可以迭代地调整在确定每一所述重投影误差时对应的第一采样点的第一全局坐标，以最小化所述重投影误差函数值，得到每一所述第一目标匹配对集合中每一第一采样点的目标全局坐标。迭代方法可以通过如下实施例中的步骤S910至步骤S914实现。
本申请实施例再一种地图融合方法，所述方法可以包括以下步骤S901至步骤S916：
步骤S901，获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
步骤S902，以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；
步骤S903，将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；
步骤S904，获取每一所述第一地图在被融合到当前第二地图中时，所确定的第一目标匹配对集合，所述第一目标匹配对集合包括所述第一地图中第i个第一采样点、和所述当前第二地图中与所述第i个第一采样点匹配的第二采样点，i为大于0的整数；
步骤S905，获取所述第一地图集合中第n个第一地图对应的投影矩阵；
步骤S906，获取第一坐标转换关系，所述第一坐标转换关系为所述第n个第一地图在被融合时相对于当前第二地图的坐标转换关系，n为大于0的整数；
步骤S907，获取第k个第一采样点的深度信息，所述第k个第一采样点为所述第n个第一地图对应的第一目标匹配对集合中的第一采样点；
步骤S908，根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点的重投影误差，k为大于0的整数；
步骤S909，根据每一所述第一目标匹配对集合中每一第一采样点的重投影误差，确定重投影误差函数值；
步骤S910，调整每一所述第一坐标转换关系，得到对应的第四坐标转换关系；
步骤S911，根据所述第k个第一采样点的局部坐标和所述第四坐标转换关系，重新确定所述第k个第一采样点的第一全局坐标；
步骤S912，根据所述第k个第一采样点的深度信息和重新确定的第一全局坐标、所述第四坐标转换关系和所述投影矩阵，重新确定所述第k个第一采样点的重投影误差；
步骤S913，根据每一所述第一目标匹配对集合中每一第一采样点的重新确定的重投影误差，重新确定重投影误差函数值；
步骤S914，如果重新确定的重投影误差函数值小于第二阈值，将所述第四全局坐标确定为所述目标全局坐标；
步骤S915，如果重新确定的重投影误差函数值大于或等于所述第二阈值，继续调整所述第四坐标转换关系，以重新确定重投影误差函数值，直到当前确定的重投影误差函数值小于所述第二阈值为止，将当前确定的第一全局坐标确定为所述目标全局坐标；
步骤S916，将每一第一采样点的目标全局坐标更新至所述初始全局地图中，得到所述目标全局地图。
在本申请实施例中，通过迭代调整每一第一坐标转换关系，寻找最优的目标全局坐标；如此，能够更加快速地确定所述第一全局坐标的最优目标全局坐标，从而提高地图融合精度。
在本申请实施例中，在未融合任一第一地图之前的第二地图，电子设备可以通过如下步骤S111至步骤S115实现对第二地图的预先构建：
步骤S111，根据多张样本图像中第一样本点的局部坐标和图像特征，确定至少一个第一样本点的全局坐标。
在实现时，可以利用图像采集模组按照特定帧率进行样本图像的采集。例如，采用单目摄像头，以固定帧率进行红、绿、蓝(Red、Green、Blue，RGB)图像的采集。或者，也可以从预先采集的样本图像库中获取所述多张样本图像。
在第二地图构建的初始阶段，可以获得第一样本点的图像特征和局部坐标，但是第一样本点的全局坐标是无从得知的。在实现时，可以通过三维重建方法对多张样本图像进行处理，从而得到第一样本点的全局坐标。例如，通过运动中恢复结构(Structure from motion，SFM)方法，对多张样本图像进行初始化处理，从而得到每一样本点的全局坐标。
步骤S112，根据每一所述第一样本点的全局坐标和图像特征，确定第一数据集；即第一数据集中包括每一所述第一样本点的全局坐标和对应的图像特征；
步骤S113，根据获取的第m张其他样本图像中第二样本点的局部坐标和图像特征，确定与所述第m张其他样本图像对应的第二数据集，m为大于0的整数；
步骤S114，根据所述第一数据集，确定所述第二数据集中第二样本点的全局坐标。
事实上，通过步骤S111确定多张样本图像中第一样本点的全局坐标，其时间复杂度是比较高的。因此，在获得第一样本点的全局坐标之后，通过步骤S114来确定第二样本点的全局坐标；如此，可以大大降低构建第二地图时的时间成本。在实现时，可以通过类似于前述实施例所提供步骤S311至步骤S313确定第二样本点的全局坐标。或者，通过类似于步骤S502至步骤S505，来确定第二样本点的全局坐标。
步骤S115，至少根据每一所述第一样本点的全局坐标和每一所述第二样本点的全局坐标，构建所述第二地图。
需要说明的是，构建的第二地图即为前述实施例中未融合任一第一地图的第二地图。第一样本点和第二样本点在第二地图中被称为第二采样点。在一些实施例中，构建的第二地图中包括第二采样点的全局坐标和图像特征。
在本申请实施例中，在进行地图构建时，首先通过多张样本图像获得第一样本点的全局坐标之后，根据每一第一样本点的全局坐标和获取的所述其他样本图像中第二样本点的局部坐标和图像特征，确定所述其他样本图像中第二样本点的全局坐标，得到第二数据集；如此，可以快速获得所述其他样本图像中样本点的全局坐标，从而降低第二地图的构建成本。
在一些实施例中，对于上述步骤S111，根据多张样本图像中第一样本点的局部坐标和图像特征，确定至少一个第一样本点的全局坐标，可以通过如下步骤S121至步骤S123实现：
步骤S121，根据每一所述第一样本点的局部坐标和图像特征，从所述多张样本图像中挑选出满足第四条件的第一目标图像和第二目标图像。
在实现时，挑选出的第一目标图像和第二目标图像一般是视差比较大的两张样本图像，这样，可以提高确定第一目标图像或第二目标图像中的样本点的全局坐标的准确度，进而有利于后续获得更好的地图融合精度。例如可以这样实现：根据每一所述第一样本点的图像特征，将所述多张样本图像进行两两匹配，得到每一对样本图像的第二匹配对集合；剔除所述第一匹配对集合中不满足第四条件的匹配对，得到第三匹配对集合；从每一所述第三匹配对集合中，挑选出匹配对数目满足所述第四条件的第二目标匹配对集合；将所述第二目标匹配对集合对应的两张样本图像，确定为第一目标图像和第二目标图像。
步骤S122，确定所述第一目标图像与所述第二目标图像之间的第五坐标转换关系。
在实现时，可以采用随机抽样一致(Random Sample Consensus，RANSAC)算法中的四点法对 所述第一目标图像和所述第二目标图像进行处理，计算单应矩阵，从而获得所述第五坐标转换关系(其中包括旋转关系和平移关系)。
步骤S123，根据所述第五坐标转换关系和所述第一目标图像中的第一样本点的局部坐标，确定所述第一目标图像中的第一样本点的全局坐标。
第一目标样本图像中的样本点与匹配的第二目标样本图像的样本点实际上同一位置点，所以，这里确定这两张目标样本图像中的任一一张图像中的样本点的全局坐标即可。
下面将说明本申请实施例在一个实际的应用场景中的示例性应用。
本申请实施例实现了一种基于稀疏点云的室内地图融合技术，可以达成多个局部地图(包括第一地图和第二地图)融合以及地图更新的目标。该方案可以支持室内多个有重复区域的局部地图合成的需求。局部地图的收集可以通过众包的形式无序地进行采集。该方案可以支持地图融合、地图更新、多人建图等日常任务，合成地图的精度高、鲁棒性强。
在本申请实施例中，局部地图是通过单目摄像头采集RGB图像信息，并从中提取图像特征而构建的稀疏点云地图。本申请实施例中，所提取的图像特征是FAST角点(即特征点)的ORB描述子信息，并确定每一角点的三维坐标信息。其中，构建第二地图的具体技术步骤至少包括以下步骤S11至步骤S15：
步骤S11，利用单目摄像头，以固定帧率进行RGB图像采集；
步骤S12，采集过程中实时提取图像中的特征信息；
步骤S13，采集一定数量图像后，利用SFM方法进行初始化；
步骤S14，初始化完成后通过PnP(Perspective-n-Point)算法计算后续图像特征点的三维坐标(即第二样本点的全局坐标)，得到第二地图；
步骤S15，存储第二地图及其对应的图像特征信息，并序列化存储到本地。
其中，针对步骤S12中，提取图像中的特征信息，这里给出如下解释。特征提取是对RGB图像的一种解释和标注的过程。这里对RGB图像提取FAST角点，提取数量固定为150个，用于图像跟踪；并对该角点进行ORB描述子的提取，用于特征点描述子匹配。需要说明的是，150为经验值，在本申请中对于提取的角点数量不做限定。角点数量过少，跟踪失败率高，角点数量过多，影响算法效率。
其中，针对步骤S13中，利用SFM方法进行初始化，这里给出如下解释。首先在采集到一定数量的图像后，通过SFM方法对图像的相对旋转和平移进行初始化，并得到特征点的三维坐标(即第一样本点的全局坐标)。SFM算法至少包括以下步骤S131至步骤S139：
步骤S131，对一定数量的图像进行两两匹配，使用欧氏距离判断的方法建立图像特征点之间的匹配关系；
步骤S132，对匹配对进行剔除，剔除方法采用RANSAC八点法计算基础矩阵，不满足基础矩阵的匹配对就选择剔除掉；
步骤S133，匹配关系建立后，生成追踪列表，追踪列表是指同名点的图像名集合；
步骤S134，剔除追踪列表中的无效匹配；
步骤S135，寻找初始化图像对(即第一目标图像和第二目标图像)，目的是找到相机基线最大的图像对，采用RANSAC算法四点法计算单应矩阵，满足单应矩阵的匹配点成为内点，不满足的成为外点。然后，找到内点占比最小的图像对。
步骤S136，寻找初始化图像对的相对旋转和平移，方法为通过RANSAC八点法计算本质矩阵，通过对本质矩阵SVD分解得到图像对之间的相对旋转和平移；
步骤S137，通过三角化计算得到初始化图像对中的特征点的三维坐标；
步骤S138，对其他图像反复执行步骤S136和步骤S137，就可以得到所有图像的相对旋转和平移，以及特征点的三维坐标；
步骤S139，通过光束平差法，优化得到的图像之间的旋转、平移和特征点的三维坐标。这是一个非线性优化的过程，目的是降低SFM结果的误差。
基于步骤S11至步骤S15可以构建出一张基于稀疏点云的离线地图，即第二地图，该地图以二进制格式存储稀疏点云及其图像特征信息(包括三维坐标和描述子信息)到本地，在视觉定位过程中，该地图将被加载使用。
在本申请实施例中，地图更新部分主要是通过描述子匹配到两个局部地图中相对应的稀疏点云，形成点云匹配对；然后再通过PnP算法求解当前第一地图相对于第二地图的精确位姿，对第一地图 进行旋转平移后，合并到第二地图以达到地图更新的目的。具体的技术步骤至少包括以下步骤S21至步骤S26：
步骤S21，加载构建好的第二地图和第一地图，以第二地图坐标系为全局坐标系；
步骤S22，在第一地图中，通过描述子匹配第二地图中的对应点，形成点云匹配对；
步骤S23，找到足够多(比如特定数量)的匹配对之后，通过PnP算法求解第一地图中的稀疏点云在全局坐标系中的精确位姿；
步骤S24，更新第一地图中的所有稀疏点云到全局坐标系中，并将第一地图的稀疏点云集合添加到第二地图的稀疏点云集合；
步骤S25，反复执行步骤S22至步骤S24，将其他局部地图融合到全局坐标系中；
步骤S26，利用优化方法优化融合后的局部地图位姿，消除多个局部地图融合的累计误差，进一步提高地图更新的精度。
其中，针对步骤S22中，在第一地图中，通过描述子匹配第二地图中的对应点，形成点云匹配对，算法具体步骤至少包括以下步骤S221至步骤S224：
步骤S221，对第一地图中的第N(初始为0)个稀疏点F 1N(即第一采样点)，设定欧式距离最小值d min＝d TH，设定匹配点
Figure PCTCN2020116926-appb-000016
步骤S222，计算F 1N和第二地图中点云中的第M(初始为0)个稀疏点F 2M，计算特征点描述子之间的欧式距离d NM；
步骤S223，判断欧式距离d NM与欧式距离最小值d min，若d NM＜d min，则d min＝d NM，
Figure PCTCN2020116926-appb-000017
接着M＝M+1，若第二地图中的稀疏点(即第二采样点)未遍历完毕，则跳转回步骤S222；否则N＝N+1，跳转回步骤S221。若第一地图中的稀疏点遍历完毕，则跳转到步骤S224；
步骤S224，整理第一地图和第二地图中的稀疏点匹配对作为算法输出，算法结束。
其中，针对步骤S23中，通过PnP算法求解第一地图中的稀疏点云在全局坐标系中的精确位姿，有一个优选实例如图3所示，在该优选实例中，算法步骤如下：
首先判断步骤S22中形成匹配对序列(在本实例中匹配对序列为{F 0,F 1,F 2})，若匹配对序列的元素数量大于TH 2，则进行步骤S23，否则算法结束，该局部点云融合失败。在该优选实例中，基于匹配对序列，调用OpenCV中的SolvePnP函数求解出当前相机在地图坐标系下的位姿。其中PnP算法的原理如下：
PnP算法的输入是第二地图中的3D点(即第二采样点)和这些3D点在第一地图中的投影得到的2D点，输出是第一地图相对于第二地图所在的全局坐标系的原点的位姿变换。
PnP算法根据匹配对序列先求出对应的2D点在第一地图所在的坐标系下的3D坐标，然后根据3D点在全局坐标系下的坐标和2D点在第一地图所在的坐标系下的3D坐标求解第一地图中稀疏点的全局坐标。
针对步骤S24中，更新第一地图中的稀疏点云到全局坐标系中，这里给出如下解释。当通过步骤S23求得第一地图中的稀疏点云在全局坐标系中的位姿时，可以通过该全局位姿和该稀疏点云在第一地图中的局部位姿，求得第二局部坐标系相对于全局坐标系的旋转矩阵和平移向量。将所有第二局部坐标系中的稀疏点，叠加上这个旋转矩阵和平移向量，即可将第一地图中的稀疏点云位姿更新到全局坐标系中。
针对步骤S26中，利用优化方法优化融合后的局部地图位姿，这里给出如下解释。在针对多个局部地图的优化方法上，通常采用最小化重投影误差的方案，将每个局部地图对应一个旋转和平移状态，通过稀疏点云的匹配在多个局部地图之间构建联系。局部地图的稀疏点匹配对在步骤S22中已经获得，每个局部地图在全局坐标系中的旋转和平移也在步骤S23中计算得到，这里综合考虑所有局部地图的位姿和稀疏点匹配对，构建重投影误差函数，然后最小化这个代价函数，即可优化局部地图的融合精度。
关于重投影误差，通过特征匹配对可知，观测值{p 1,p 2}是一组特征匹配对，它们是同一个空间点P的投影，如图4所示，p 1属于第一地图，p 2属于第二地图，
Figure PCTCN2020116926-appb-000018
为将p 1转换为在第二地图中的第一全局坐标之后在第二地图上的投影点。
P的投影
Figure PCTCN2020116926-appb-000019
与观测值p 2之间有一定的距离，这就是重投影误差。当调整局部点云的位姿，可以 使这个距离变小，这就起到了对局部点云位姿的优化效果，由于这个调整需要考虑很多个点，所以最后每个点的误差通常不会都为0。考虑N个三维空间点P和它们的投影
Figure PCTCN2020116926-appb-000020
计算每个局部点云的旋转R和平移T，用李代数表示为ξ。假设某个空间点P k＝[X k,Y k,Z k] T，其投影坐标为U k＝[u k,v k] T，两者之间的关系如下式(12)所示：
Figure PCTCN2020116926-appb-000021
矩阵形式如下式(13)所示：
s kU k＝K exp(ξ)P k     (13)；
式中，s k表示空间点P k在第一地图中的深度信息，K表示第一地图的投影矩阵。由于局部地图位姿误差以及观测点的噪声，该等式存在一个重投影误差，因此，如公式(14)所示，将重投影误差求和，构建最小二乘问题，寻找最好的局部地图位姿，使等式最小化：
Figure PCTCN2020116926-appb-000022
式中，μ k空间点P k在第二地图中的像素坐标，
Figure PCTCN2020116926-appb-000023
表示空间点P k的重投影误差，以图4为例，该投影误差为投影
Figure PCTCN2020116926-appb-000024
与观测值p 2之间的像素距离。在实现时，可以用高斯牛顿法或者列文伯格-马夸特方法可以求解这个最小二乘问题，由此即可求得优化之后的局部地图位姿。
基于步骤S21至步骤S26可以通过视觉特征，达成对多个预定义的稀疏点云局部地图进行融合、更新的目的。该地图更新方案融合精度较高，鲁棒性强。
在本申请实施例中，通过使用最优化理论，减少地图融合的累计误差，提升合成后的全局地图一致性。
在本申请实施例中，结合视觉特征的三维信息，采用了高精度高鲁棒性的匹配算法，在地图融合的结果上相对于其他室内定位方法提高了融合精度。
在本申请实施例中，不需要两个局部地图有大量的重复区域，也能保证良好的地图融合精度，在以众包进行地图更新或多人建图的时候，提升了地图融合的稳定性，同时也提升了构建局部地图的效率。
在本申请实施例中，存储的地图形式为稀疏点云，相当于图像的稀疏采样，在地图大小上较之传统方法有一定程度的压缩。
在本申请实施例中，充分挖掘图像特征的三维信息，并结合优化理论达成了地图融合的目标。在地图构建上，本申请实施例通过采集视觉图像中特征点的三维坐标和描述子信息，以稀疏点云的形式存储为离线地图。在地图融合方法上，本申请实施例采用描述子匹配的方法找到当前特征点在稀疏点云中的匹配对；再通过PnP算法精确地计算出当前局部地图在全局坐标系中的位置和姿态；最后结合优化理论，进而达成地图融合目的，形成了一套融合准确度高、鲁棒性强的地图融合方案。该方案支持多人建图场景下的地图融合，以及众包形式的地图更新，在保证地图融合稳定性的同时也提升了构建局部地图的效率。
在地图构建部分，本申请实施例只序列化存储稀疏点云的三维坐标信息。
在地图融合部分，本申请实施例提出了一种调整后的地图融合方案，技术步骤至少包括以下步骤S31至步骤S35：
步骤S31，加载构建好的第二地图和第一地图，以第二地图坐标系为全局坐标系；
步骤S32，通过迭代最近点算法(Iterative Closest Point，ICP)求解第一地图稀疏点云相对于第二地图稀疏点云的旋转和平移；
步骤S33，更新第一地图中的所有稀疏点云到全局坐标系中，并将第一地图的稀疏点云集合添加到第二地图的稀疏点云集合；
步骤S34，反复执行步骤S32和步骤S33，将其他局部地图融合到全局坐标系中；
步骤S35，利用优化方法优化融合后的局部地图位姿，消除多个局部地图融合的累计误差，进一 步提高地图更新的精度。
其中，针对步骤S32中，通过ICP算法求解第一地图稀疏点云相对于第二地图稀疏点云的旋转和平移，这里给出如下解释。ICP算法，其本质上是基于最小二乘法的最优配准方法。该算法重复进行选择对应关系点对，计算最优刚体变换，直到满足正确配准的收敛精度要求。ICP算法的基本原理是：分别在待匹配的目标点云P和源点云Q中，按照一定的约束条件，找到最邻近的点(p l,q l)，然后计算出最优的旋转R和平移T，使得误差函数最小，误差函数E(R,T)为
Figure PCTCN2020116926-appb-000025
其中h为邻近点对的数量，p l为目标点云P中的一点，q l为源点云Q中与p l对应的最近点，R为旋转矩阵，T为平移向量。
基于步骤S31至步骤S35可以通过视觉特征达成融合多个预定义的稀疏点云地图的目的，且预定的稀疏点云地图中不需要存储额外的特征点描述子信息，压缩了离线地图的大小。
基于前述的实施例，本申请实施例提供一种地图融合装置，该装置包括所包括的各模块、以及各模块所包括的各单元，可以通过电子设备中的处理器来实现；当然也可通过具体的逻辑电路实现；在实施的过程中，处理器可以为中央处理器(CPU)、微处理器(MPU)、数字信号处理器(DSP)或现场可编程门阵列(FPGA)等。
图5A为本申请实施例地图融合装置的组成结构示意图，如图5A所示，所述装置500包括地图获取模块501、坐标转换模块502、地图融合模块503和地图优化模块504，其中：地图获取模块501，配置为获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；坐标转换模块502，配置为以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；地图融合模块503，配置为将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；地图优化模块504，配置为对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
在一些实施例中，坐标转换模块502，配置为：以所述第二地图的坐标系为全局坐标系，依次将所述第一地图集合中满足条件的第一地图中第一采样点的局部坐标，转换为第一全局坐标；地图融合模块503，配置为：依次将所述第一地图集合中满足条件的第一地图中第一采样点的第一全局坐标，融合到所述第二地图中，得到所述初始全局地图。
在一些实施例中，坐标转换模块502，包括：确定子模块和坐标转换子模块；其中，所述确定子模块，配置为：将所述第一地图集合中与所述第二地图相匹配的第一地图，确定为目标局部地图；所述坐标转换子模块，配置为：将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标；地图融合模块503，配置为：将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图。
在一些实施例中，所述确定子模块，包括：特征匹配单元，配置为将每一所述第一地图中第一采样点的图像特征，分别与所述第二地图中第二采样点的图像特征进行匹配，得到与所述第一地图对应的第一匹配对集合；第一确定单元，配置为将满足第一条件的第一匹配对集合，确定为第一目标匹配对集合；将与所述第一目标匹配对集合对应的第一地图，确定为所述目标局部地图。
在一些实施例中，所述特征匹配单元，配置为：确定所述第一地图中第i个第一采样点的图像特征与每一所述第二采样点之间的相似度，得到相似度集合，i为大于0的整数；将所述相似度集合中相似度满足第二条件的第二采样点，确定为目标点；将每一所述目标点和对应的第一采样点进行关联，得到与所述第一地图对应的第一匹配对集合。
在一些实施例中，所述坐标转换子模块，包括：第二确定单元，配置为：从所述第二地图的第二采样点中，确定出与所述目标局部地图中第一采样点的图像特征相匹配的目标点；根据所述目标局部地图中每一第一采样点的局部坐标和对应的目标点的全局坐标，确定所述目标局部地图相对于所述第二地图的第一坐标转换关系；坐标转换单元，配置为：根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
在一些实施例中，所述第二确定单元，配置为：根据所述目标局部地图中每一第一采样点的局 部坐标和对应的目标点的全局坐标，确定每一所述目标点在所述目标局部地图中的局部坐标；根据每一所述目标点的全局坐标和局部坐标，确定所述第一坐标转换关系。
在一些实施例中，所述确定子模块，包括：迭代单元和第三确定单元；其中，所述迭代单元，配置为根据迭代策略，将所述第一地图集合中第n个第一地图的每一第一采样点的局部坐标与所述第二地图中多个第二采样点的全局坐标进行匹配，得到匹配结果，n为大于0的整数；所述第三确定单元，配置为在所述匹配结果表征匹配成功的情况下，将所述第n个第一地图确定为所述目标局部地图；所述迭代单元，还配置为在所述匹配结果表征匹配失败的情况下，继续将下一个第一地图中每一第一采样点的局部坐标与所述多个第二采样点的全局坐标进行匹配，直到从所述第一地图集合中确定出所述目标局部地图为止。
在一些实施例中，所述迭代单元，包括：选取子单元，配置为从所述多个第二采样点中，选取与所述第n个第一地图中每一第一采样点匹配的初始目标点；确定子单元，配置为根据所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定所述第n个第一地图相对于所述第二地图的第二坐标转换关系；根据所述第二坐标转换关系、所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定匹配误差；如果所述匹配误差大于第一阈值，重新选取初始目标点，并重新确定匹配误差；结果生成单元，配置为如果所述匹配误差小于或等于所述第一阈值，生成表征匹配成功的匹配结果。
在一些实施例中，所述迭代单元，还配置为：如果确定匹配误差的次数大于第二阈值，生成表征匹配失败的匹配结果，并继续从所述多个第二采样点中，选取与下一个第一地图中每一第一采样点匹配的初始目标点，直到生成表征匹配成功的匹配结果为止。
在一些实施例中所述坐标转换单元，配置为：在所述匹配结果表征匹配成功的情况下，将确定的匹配误差小于或等于所述第一阈值时的第二坐标转换关系，确定为所述目标局部地图相对于所述第二地图的第一坐标转换关系；根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
在一些实施例中，所述选取子单元，配置为：获取所述第n个第一地图相对于所述第二地图的第三坐标转换关系；根据所述第三坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第二全局坐标，j为大于0的整数；将所述第二全局坐标与所述多个第二采样点的全局坐标进行匹配，得出与所述第j个第一采样点匹配的初始目标点。
在一些实施例中，所述确定子单元，配置为：根据第二坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第三全局坐标，j为大于0的整数；根据所述第n个第一地图中每一第一采样点的第三全局坐标和对应的初始目标点的全局坐标，确定所述匹配误差。
在一些实施例中，所述确定子单元，配置为：确定所述第n个第一地图中每一第一采样点的第三全局坐标与对应的初始目标点的全局坐标之间的距离；根据每一所述距离，确定所述匹配误差。
在一些实施例中，确定子单元，配置为如果所述匹配误差大于所述第一阈值，将所述第二坐标转换关系作为所述第三坐标转换关系，重新选取初始目标点。
在一些实施例中，所述地图优化模块504，包括：获取子模块，配置为获取每一所述第一地图在被融合到当前第二地图中时，所确定的第一目标匹配对集合，所述第一目标匹配对集合包括所述第一地图中第i个第一采样点、和所述当前第二地图中与所述第i个第一采样点匹配的第二采样点，i为大于0的整数；坐标优化子模块，配置为对每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标进行优化，得到对应的第一采样点的目标全局坐标；更新子模块，配置为将每一第一采样点的目标全局坐标更新至所述初始全局地图中，得到所述目标全局地图。
在一些实施例中，所述坐标优化子模块，包括：第三确定单元，配置为根据每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标，确定对应的第一采样点的重投影误差；坐标调整单元，配置为调整每一所述第一目标匹配对集合中每一第一采样点的第一全局坐标，使得每一第一采样点的重投影误差满足第三条件，从而得到对应的第一采样点的目标全局坐标。
在一些实施例中，所述第三确定单元，配置为：获取所述第一地图集合中第n个第一地图对应的投影矩阵；获取第一坐标转换关系，所述第一坐标转换关系为所述第n个第一地图在被融合时相对于当前第二地图的坐标转换关系，n为大于0的整数；获取第k个第一采样点的深度信息，所述第k个第一采样点为所述第n个第一地图对应的第一目标匹配对集合中的第一采样点；根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k 个第一采样点的重投影误差，k为大于0的整数。
在一些实施例中，所述第三确定单元，配置为：根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点在所述当前第二地图中的投影坐标；根据所述投影坐标和与所述第k个第一采样点匹配的目标点的像素坐标，确定所述第k个第一采样点的重投影误差。
在一些实施例中，所述坐标调整单元，配置为：根据每一所述第一目标匹配对集合中每一第一采样点的重投影误差，确定重投影误差函数值；迭代地调整在确定每一所述重投影误差时对应的第一采样点的第一全局坐标，以最小化所述重投影误差函数值，得到每一所述第一目标匹配对集合中每一第一采样点的目标全局坐标。
在一些实施例中，所述坐标调整单元，配置为：调整每一所述第一坐标转换关系，得到对应的第四坐标转换关系；根据所述第k个第一采样点的局部坐标和所述第四坐标转换关系，重新确定所述第k个第一采样点的第一全局坐标；根据所述第k个第一采样点的深度信息和重新确定的第一全局坐标、所述第四坐标转换关系和所述投影矩阵，重新确定所述第k个第一采样点的重投影误差；根据每一所述第一目标匹配对集合中每一第一采样点的重新确定的重投影误差，重新确定重投影误差函数值；如果重新确定的重投影误差函数值小于第二阈值，将所述第四全局坐标确定为所述目标全局坐标。
在一些实施例中，所述坐标调整单元，还配置为：如果重新确定的重投影误差函数值大于或等于所述第二阈值，继续调整所述第四坐标转换关系，以重新确定重投影误差函数值，直到当前确定的重投影误差函数值小于所述第二阈值为止，将当前确定的第一全局坐标确定为所述目标全局坐标。
在一些实施例中，如图5B所示，所述装置500还包括：确定模块505，配置为：根据多张样本图像中第一样本点的局部坐标和图像特征，确定至少一个第一样本点的全局坐标；根据每一所述第一样本点的全局坐标和图像特征，确定第一数据集；根据获取的第m张其他样本图像中第二样本点的局部坐标和图像特征，确定与所述第m张其他样本图像对应的第二数据集，m为大于0的整数；根据所述第一数据集，确定所述第二数据集中第二样本点的全局坐标；地图构建模块，配置为至少根据每一所述第一样本点的全局坐标和每一所述第二样本点的全局坐标，构建所述第二地图。
在一些实施例中，所述确定模块505，配置为：根据每一所述第一样本点的局部坐标和图像特征，从所述多张样本图像中挑选出满足第四条件的第一目标图像和第二目标图像；确定所述第一目标图像与所述第二目标图像之间的第五坐标转换关系；根据所述第五坐标转换关系和所述第一目标图像中的第一样本点的局部坐标，确定所述第一目标图像中的第一样本点的全局坐标。
在一些实施例中，所述确定模块505，配置为：根据每一所述第一样本点的图像特征，将所述多张样本图像进行两两匹配，得到每一对样本图像的第二匹配对集合；剔除所述第一匹配对集合中不满足第四条件的匹配对，得到第三匹配对集合；从每一所述第三匹配对集合中，挑选出匹配对数目满足所述第四条件的第二目标匹配对集合；将所述第二目标匹配对集合对应的两张样本图像，确定为第一目标图像和第二目标图像。
以上装置实施例的描述与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请装置实施例中未披露的技术细节，参照本申请方法实施例的描述而理解。
需要说明的是，本申请实施例中，如果以软件功能模块的形式实现上述的地图融合方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得电子设备(可以是手机、平板电脑、笔记本电脑、台式计算机、机器人、无人机、服务器等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本申请实施例不限制于任何特定的硬件和软件结合。
对应地，本申请实施例提供一种电子设备，图6为本申请实施例电子设备的一种硬件实体示意图，如图6所示，该电子设备600的硬件实体包括：包括存储器601和处理器602，所述存储器601存储有可在处理器602上运行的计算机程序，所述处理器602执行所述程序时实现上述实施例中提供的地图融合方法中的步骤。
存储器601配置为存储由处理器602可执行的指令和应用，还可以缓存待处理器602以及电子设备600中各模块待处理或已经处理的数据(例如，图像数据、音频数据、语音通信数据和视频通信数据)，可以通过闪存(FLASH)或随机访问存储器(Random Access Memory，RAM)实现。
对应地，本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述实施例中提供的地图融合方法中的步骤。
这里需要指出的是：以上存储介质和设备实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请存储介质和设备实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
应理解，说明书通篇中提到的“一个实施例”或“一些实施例”意味着与实施例有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一些实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如对象A和/或对象B，可以表示：单独存在对象A，同时存在对象A和对象B，单独存在对象B这三种情况。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本申请各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。或者，本申请上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得电子设备(可以是手机、平板电脑、笔记本电脑、台式计算机、机器人、无人机、服务器等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。
本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。本申请所提供的几个产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
以上所述，仅为本申请的实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (28)

  1. 一种地图融合方法，所述方法包括：
    获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
    以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；
    将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；
    对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
  2. 根据权利要求1所述的方法，其中，所述以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标，包括：
    以所述第二地图的坐标系为全局坐标系，依次将所述第一地图集合中满足条件的第一地图中第一采样点的局部坐标，转换为第一全局坐标；
    所述将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图，包括：
    依次将所述第一地图集合中满足条件的第一地图中第一采样点的第一全局坐标，融合到当前第二地图中，得到所述初始全局地图。
  3. 根据权利要求2所述的方法，其中，所述依次将所述第一地图集合中满足条件的第一地图中第一采样点的局部坐标，转换为第一全局坐标，包括：将所述第一地图集合中与所述第二地图相匹配的第一地图，确定为目标局部地图；将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标；
    所述依次将所述第一地图集合中满足条件的第一地图中第一采样点的第一全局坐标，融合到所述第二地图中，得到所述初始全局地图，包括：
    将所述目标局部地图中第一采样点的第一局部坐标融合到所述第二地图中，得到融合后的第二地图；
    从所述第一地图集合中剩余的第一地图中，确定与所述融合后的第二地图相匹配的新的目标局部地图，以将所述新的目标局部地图中第一采样点的第一全局坐标，融合到所述融合后的第二地图中，直到每一所述第一地图均被融合到所述第二地图中为止，得到所述初始全局地图。
  4. 根据权利要求3所述的方法，其中，所述将所述第一地图集合中与所述第二地图相匹配的第一地图，确定为目标局部地图，包括：
    将每一所述第一地图中第一采样点的图像特征，分别与所述第二地图中第二采样点的图像特征进行匹配，得到与所述第一地图对应的第一匹配对集合；
    将满足第一条件的第一匹配对集合，确定为第一目标匹配对集合；
    将与所述第一目标匹配对集合对应的第一地图，确定为所述目标局部地图。
  5. 根据权利要求4所述的方法，其中，所述将每一所述第一地图中第一采样点的图像特征，分别与所述第二地图中第二采样点的图像特征进行匹配，得到与所述第一地图对应的第一匹配对集合，包括：
    确定所述第一地图中第i个第一采样点的图像特征与每一所述第二采样点之间的相似度，得到相似度集合，i为大于0的整数；
    将所述相似度集合中相似度满足第二条件的第二采样点，确定为目标点；
    将每一所述目标点和对应的第一采样点进行关联，得到与所述第一地图对应的第一匹配对集合。
  6. 根据权利要求3所述的方法，其中，所述将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标，包括：
    从所述第二地图的第二采样点中，确定出与所述目标局部地图中第一采样点的图像特征相匹配的目标点；
    根据所述目标局部地图中多个第一采样点的局部坐标和与所述多个第一采样点分别对应的目标点的全局坐标，确定所述目标局部地图相对于所述第二地图的第一坐标转换关系；
    根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐 标。
  7. 根据权利要求6所述的方法，其中，所述根据所述目标局部地图中多个第一采样点的局部坐标和与所述多个第一采样点分别对应的目标点的全局坐标，确定所述目标局部地图相对于所述第二地图的第一坐标转换关系，包括：
    根据所述多个第一采样点的局部坐标和与所述多个第一采样点分别对应的目标点的全局坐标，确定与所述多个第一采样点分别对应的目标点在所述目标局部地图中的局部坐标；
    根据每一目标点的全局坐标和局部坐标，确定所述第一坐标转换关系。
  8. 根据权利要求3所述的方法，其中，所述将所述第一地图集合中与所述第二地图相匹配的第一地图，确定为目标局部地图，包括：
    根据迭代策略，将所述第一地图集合中第n个第一地图的每一第一采样点的局部坐标，与所述第二地图中多个第二采样点的全局坐标进行匹配，得到匹配结果，n为大于0的整数；
    在所述匹配结果表征匹配成功的情况下，将所述第n个第一地图确定为所述目标局部地图；
    在所述匹配结果表征匹配失败的情况下，继续将下一个第一地图中每一第一采样点的局部坐标与所述多个第二采样点的全局坐标进行匹配，直到从所述第一地图集合中确定出所述目标局部地图为止。
  9. 根据权利要求8所述的方法，其中，所述根据迭代策略，将所述第一地图集合中第n个第一地图的每一第一采样点的局部坐标，与所述第二地图中多个第二采样点的全局坐标进行匹配，得到匹配结果，包括：
    从所述多个第二采样点中，选取与所述第n个第一地图中每一第一采样点匹配的初始目标点；
    根据所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定所述第n个第一地图相对于所述第二地图的第二坐标转换关系；
    根据所述第二坐标转换关系、所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定匹配误差；
    如果所述匹配误差大于第一阈值，重新选取初始目标点，并重新确定匹配误差；
    如果所述匹配误差小于或等于所述第一阈值，生成表征匹配成功的匹配结果。
  10. 根据权利要求9所述的方法，其中，所述方法还包括：
    如果确定匹配误差的次数大于第二阈值，生成表征匹配失败的匹配结果，并继续从所述多个第二采样点中，选取与下一个第一地图中每一第一采样点匹配的初始目标点，直到生成表征匹配成功的匹配结果为止。
  11. 根据权利要求9所述的方法，其中，所述将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标，包括：
    在所述匹配结果表征匹配成功的情况下，将确定的匹配误差小于或等于所述第一阈值时的第二坐标转换关系，确定为所述目标局部地图相对于所述第二地图的第一坐标转换关系；
    根据所述第一坐标转换关系，将所述目标局部地图中第一采样点的局部坐标转换为第一全局坐标。
  12. 根据权利要求9所述的方法，其中，所述从所述多个第二采样点中，选取与所述第n个第一地图中每一第一采样点匹配的初始目标点，包括：
    获取所述第n个第一地图相对于所述第二地图的第三坐标转换关系；
    根据所述第三坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第二全局坐标，j为大于0的整数；
    将所述第二全局坐标与所述多个第二采样点的全局坐标进行匹配，得出与所述第j个第一采样点匹配的初始目标点。
  13. 根据权利要求9所述的方法，其中，所述根据所述第二坐标转换关系、所述第n个第一地图中每一第一采样点的局部坐标和对应的初始目标点的全局坐标，确定匹配误差，包括：
    根据第二坐标转换关系和所述第n个第一地图中第j个第一采样点的局部坐标，确定所述第j个第一采样点的第三全局坐标，j为大于0的整数；
    根据所述第n个第一地图中每一第一采样点的第三全局坐标和对应的初始目标点的全局坐标，确定所述匹配误差。
  14. 根据权利要求13所述的方法，其中，所述根据所述第n个第一地图中每一第一采样点的第三全局坐标和对应的初始目标点的全局坐标，确定所述匹配误差，包括：
    确定所述第n个第一地图中每一第一采样点的第三全局坐标与对应的初始目标点的全局坐标之间的距离；
    根据每一所述距离，确定所述匹配误差。
  15. 根据权利要求12所述的方法，其中，如果所述匹配误差大于第一阈值，重新选取初始目标点，包括：
    如果所述匹配误差大于所述第一阈值，将所述第二坐标转换关系作为所述第三坐标转换关系，重新选取初始目标点。
  16. 根据权利要求1至15任一项所述的方法，其中，所述对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图，包括：
    获取每一所述第一地图在被融合到当前第二地图中时，所确定的第一目标匹配对集合，所述第一目标匹配对集合包括所述第一地图中第i个第一采样点、和所述当前第二地图中与所述第i个第一采样点匹配的第二采样点，i为大于0的整数；
    对每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标进行优化，得到对应的第一采样点的目标全局坐标；
    将每一第一采样点的目标全局坐标更新至所述初始全局地图中，得到所述目标全局地图。
  17. 根据权利要求16所述的方法，其中，所述对每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标进行优化，得到对应的第一采样点的目标全局坐标，包括：
    根据每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标，确定对应的第一采样点的重投影误差；
    调整每一所述第一目标匹配对集合中每一第一采样点的第一全局坐标，使得每一第一采样点的重投影误差满足第三条件，从而得到对应的第一采样点的目标全局坐标。
  18. 根据权利要求17所述的方法，其中，所述根据每一所述第一目标匹配对集合中的每一第一采样点的第一全局坐标，确定对应的第一采样点的重投影误差，包括：
    获取所述第一地图集合中第n个第一地图对应的投影矩阵；
    获取第一坐标转换关系，所述第一坐标转换关系为所述第n个第一地图在被融合时相对于当前第二地图的坐标转换关系，n为大于0的整数；
    获取第k个第一采样点的深度信息，所述第k个第一采样点为所述第n个第一地图对应的第一目标匹配对集合中的第一采样点；
    根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点的重投影误差，k为大于0的整数。
  19. 根据权利要求18所述的方法，其中，所述根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点的重投影误差，包括：
    根据所述投影矩阵、所述第一坐标转换关系、所述第k个第一采样点的第一全局坐标和所述深度信息，确定所述第k个第一采样点在所述当前第二地图中的投影坐标；
    根据所述投影坐标和与所述第k个第一采样点匹配的目标点的像素坐标，确定所述第k个第一采样点的重投影误差。
  20. 根据权利要求18所述的方法，其中，所述调整每一所述第一目标匹配对集合中每一第一采样点的第一全局坐标，使得每一第一采样点的重投影误差满足第三条件，从而得到对应的第一采样点的目标全局坐标，包括：
    根据每一所述第一目标匹配对集合中每一第一采样点的重投影误差，确定重投影误差函数值；
    迭代地调整在确定每一所述重投影误差时对应的第一采样点的第一全局坐标，以最小化所述重投影误差函数值，得到每一所述第一目标匹配对集合中每一第一采样点的目标全局坐标。
  21. 根据权利要求20所述的方法，其中，所述迭代地调整在确定每一所述重投影误差时对应的第一采样点的第一全局坐标，以最小化所述重投影误差函数值，得到每一所述第一目标匹配对集合中每一第一采样点的目标全局坐标，包括：
    调整每一所述第一坐标转换关系，得到对应的第四坐标转换关系；
    根据所述第k个第一采样点的局部坐标和所述第四坐标转换关系，重新确定所述第k个第一采样点的第一全局坐标；
    根据所述第k个第一采样点的深度信息和重新确定的第一全局坐标、所述第四坐标转换关系和 所述投影矩阵，重新确定所述第k个第一采样点的重投影误差；
    根据每一所述第一目标匹配对集合中每一第一采样点的重新确定的重投影误差，重新确定重投影误差函数值；
    如果重新确定的重投影误差函数值小于第二阈值，将所述第四全局坐标确定为所述目标全局坐标。
  22. 根据权利要求21所述的方法，其中，所述方法还包括：
    如果重新确定的重投影误差函数值大于或等于所述第二阈值，继续调整所述第四坐标转换关系，以重新确定重投影误差函数值，直到当前确定的重投影误差函数值小于所述第二阈值为止，将当前确定的第一全局坐标确定为所述目标全局坐标。
  23. 根据权利要求1至15任一项所述的方法，其中，所述第二地图的构建过程包括：
    根据多张样本图像中第一样本点的局部坐标和图像特征，确定至少一个第一样本点的全局坐标；
    根据每一所述第一样本点的全局坐标和图像特征，确定第一数据集；
    根据获取的第m张其他样本图像中第二样本点的局部坐标和图像特征，确定与所述第m张其他样本图像对应的第二数据集，m为大于0的整数；
    根据所述第一数据集，确定所述第二数据集中第二样本点的全局坐标；
    至少根据每一所述第一样本点的全局坐标和每一所述第二样本点的全局坐标，构建所述第二地图。
  24. 根据权利要求23所述的方法，其中，所述根据多张样本图像中第一样本点的局部坐标和图像特征，确定至少一个第一样本点的全局坐标，包括：
    根据每一所述第一样本点的局部坐标和图像特征，从所述多张样本图像中挑选出满足第四条件的第一目标图像和第二目标图像；
    确定所述第一目标图像与所述第二目标图像之间的第五坐标转换关系；
    根据所述第五坐标转换关系和所述第一目标图像中的第一样本点的局部坐标，确定所述第一目标图像中的第一样本点的全局坐标。
  25. 根据权利要求24所述的方法，其中，所述根据每一所述第一样本点的局部坐标和图像特征，从所述多张样本图像中挑选出满足第四条件的第一目标图像和第二目标图像，包括：
    根据每一所述第一样本点的图像特征，将所述多张样本图像进行两两匹配，得到每一对样本图像的第二匹配对集合；
    剔除所述第一匹配对集合中不满足第四条件的匹配对，得到第三匹配对集合；
    从每一所述第三匹配对集合中，挑选出匹配对数目满足所述第四条件的第二目标匹配对集合；
    将所述第二目标匹配对集合对应的两张样本图像，确定为第一目标图像和第二目标图像。
  26. 一种地图融合装置，包括：
    地图获取模块，配置为获取第一地图集合和第二地图，所述第一地图集合中包括一个或多个不同的第一地图，每一所述第一地图的坐标系与所述第二地图的坐标系不同；
    坐标转换模块，配置为以所述第二地图的坐标系为全局坐标系，将每一所述第一地图中第一采样点的局部坐标转换到所述全局坐标系中，得到所述第一采样点的第一全局坐标；
    地图融合模块，配置为将所述第一采样点的第一全局坐标融合到所述第二地图中，得到初始全局地图；
    地图优化模块，配置为对所述初始全局地图中所述第一采样点的第一全局坐标进行优化，以减小融合误差，得到目标全局地图。
  27. 一种电子设备，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现权利要求1至25任一项所述地图融合方法中的步骤。
  28. 一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现权利要求1至25任一项所述地图融合方法中的步骤。
PCT/CN2020/116926 2019-09-27 2020-09-22 地图融合方法及装置、设备、存储介质 WO2021057743A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910923582.3A CN110704563B (zh) 2019-09-27 2019-09-27 地图融合方法及装置、设备、存储介质
CN201910923582.3 2019-09-27

Publications (1)

Publication Number Publication Date
WO2021057743A1 true WO2021057743A1 (zh) 2021-04-01

Family

ID=69197861

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/116926 WO2021057743A1 (zh) 2019-09-27 2020-09-22 地图融合方法及装置、设备、存储介质

Country Status (2)

Country Link
CN (1) CN110704563B (zh)
WO (1) WO2021057743A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113535875A (zh) * 2021-07-14 2021-10-22 北京百度网讯科技有限公司 地图数据扩充方法、装置、电子设备、介质和程序产品
CN114255267A (zh) * 2021-12-20 2022-03-29 北京房江湖科技有限公司 用于配准户型图的方法和装置
CN114663276A (zh) * 2022-03-25 2022-06-24 北京百度网讯科技有限公司 地图数据的压缩方法、装置、设备及存储介质
CN114840703A (zh) * 2022-03-30 2022-08-02 高德软件有限公司 位姿信息获取方法、装置、设备、介质及产品
CN116481516A (zh) * 2023-06-21 2023-07-25 深圳市普渡科技有限公司 机器人、地图创建方法和存储介质
CN116757936A (zh) * 2023-08-22 2023-09-15 佗道医疗科技有限公司 图像匹配关系获取方法及其图像拼接方法
CN117314735A (zh) * 2023-09-26 2023-12-29 长光辰英(杭州)科学仪器有限公司 基于最小化重投影误差的全局优化坐标映射转换方法
CN118067114A (zh) * 2024-04-24 2024-05-24 成都赛力斯科技有限公司 一种地图构建方法、装置、电子设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704563B (zh) * 2019-09-27 2022-06-28 Oppo广东移动通信有限公司 地图融合方法及装置、设备、存储介质
CN110704562B (zh) * 2019-09-27 2022-07-19 Oppo广东移动通信有限公司 地图融合方法及装置、设备、存储介质
CN112597787B (zh) * 2020-08-27 2021-10-15 禾多科技（北京）有限公司 用于融合局部图像的方法、装置、服务器和介质
CN113506459B (zh) * 2021-06-11 2023-03-28 上海追势科技有限公司 一种地下停车场众包地图采集方法
CN113838201B (zh) * 2021-09-23 2022-06-07 北京百度网讯科技有限公司 模型适配方法、装置、电子设备及可读存储介质
CN114370865B (zh) * 2022-01-14 2024-06-04 深圳鹏行智能研究有限公司 对室内地图进行坐标转换的方法、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109559277A (zh) * 2018-11-28 2019-04-02 中国人民解放军国防科技大学 一种面向数据共享的多无人机协同地图构建方法
CN110276826A (zh) * 2019-05-23 2019-09-24 全球能源互联网研究院有限公司 一种电网作业环境地图的构建方法及系统
CN110704563A (zh) * 2019-09-27 2020-01-17 Oppo广东移动通信有限公司 地图融合方法及装置、设备、存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101984463A (zh) * 2010-11-02 2011-03-09 中兴通讯股份有限公司 全景图合成方法及装置
CN107862720B (zh) * 2017-11-24 2020-05-22 北京华捷艾米科技有限公司 基于多地图融合的位姿优化方法及位姿优化系统
CN109341706B (zh) * 2018-10-17 2020-07-03 张亮 一种面向无人驾驶汽车的多特征融合地图的制作方法
CN109781114A (zh) * 2019-02-15 2019-05-21 上海扩博智能技术有限公司 商场室内导航地图建立方法、系统、设备及存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109559277A (zh) * 2018-11-28 2019-04-02 中国人民解放军国防科技大学 一种面向数据共享的多无人机协同地图构建方法
CN110276826A (zh) * 2019-05-23 2019-09-24 全球能源互联网研究院有限公司 一种电网作业环境地图的构建方法及系统
CN110704563A (zh) * 2019-09-27 2020-01-17 Oppo广东移动通信有限公司 地图融合方法及装置、设备、存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YANG, SHAOWU: "CORB-SLAM: a Collaborative Visual SLAM System for Multiple Robots", CONFERENCE PAPER, 31 December 2017 (2017-12-31), pages 1 - 10, XP055794802 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113535875A (zh) * 2021-07-14 2021-10-22 北京百度网讯科技有限公司 地图数据扩充方法、装置、电子设备、介质和程序产品
CN114255267A (zh) * 2021-12-20 2022-03-29 北京房江湖科技有限公司 用于配准户型图的方法和装置
CN114663276A (zh) * 2022-03-25 2022-06-24 北京百度网讯科技有限公司 地图数据的压缩方法、装置、设备及存储介质
CN114840703A (zh) * 2022-03-30 2022-08-02 高德软件有限公司 位姿信息获取方法、装置、设备、介质及产品
CN116481516A (zh) * 2023-06-21 2023-07-25 深圳市普渡科技有限公司 机器人、地图创建方法和存储介质
CN116481516B (zh) * 2023-06-21 2023-10-13 深圳市普渡科技有限公司 机器人、地图创建方法和存储介质
CN116757936A (zh) * 2023-08-22 2023-09-15 佗道医疗科技有限公司 图像匹配关系获取方法及其图像拼接方法
CN116757936B (zh) * 2023-08-22 2023-11-07 佗道医疗科技有限公司 图像匹配关系获取方法及其图像拼接方法
CN117314735A (zh) * 2023-09-26 2023-12-29 长光辰英(杭州)科学仪器有限公司 基于最小化重投影误差的全局优化坐标映射转换方法
CN117314735B (zh) * 2023-09-26 2024-04-05 长光辰英(杭州)科学仪器有限公司 基于最小化重投影误差的全局优化坐标映射转换方法
CN118067114A (zh) * 2024-04-24 2024-05-24 成都赛力斯科技有限公司 一种地图构建方法、装置、电子设备及存储介质
CN118067114B (zh) * 2024-04-24 2024-07-26 成都赛力斯科技有限公司 一种地图构建方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN110704563A (zh) 2020-01-17
CN110704563B (zh) 2022-06-28

Similar Documents

Publication Publication Date Title
WO2021057743A1 (zh) 地图融合方法及装置、设备、存储介质
WO2021057742A1 (zh) 定位方法及装置、设备、存储介质
WO2021057744A1 (zh) 定位方法及装置、设备、存储介质
WO2021057745A1 (zh) 地图融合方法及装置、设备、存储介质
CN110728717B (zh) 定位方法及装置、设备、存储介质
JP7453470B2 (ja) ３次元再構成及び関連インタラクション、測定方法及び関連装置、機器
JP6745328B2 (ja) 点群データを復旧するための方法及び装置
WO2022121640A1 (zh) 机器人重定位方法、装置、机器人和可读存储介质
WO2019205852A1 (zh) 确定图像捕捉设备的位姿的方法、装置及其存储介质
EP3698275A1 (en) Data processing method, apparatus, system and storage media
WO2020087564A1 (zh) 三维物体重建方法、计算机设备及存储介质
JP7546151B2 (ja) 動画移行方法及び装置、機器、記憶媒体並びにコンピュータプログラム
CN111784776B (zh) 视觉定位方法及装置、计算机可读介质和电子设备
WO2020259361A1 (zh) 地图更新方法及装置、终端、存储介质
WO2023151251A1 (zh) 地图构建方法、位姿确定方法、装置、设备及计算机程序产品
GB2566443A (en) Cross-source point cloud registration
WO2023221790A1 (zh) 图像编码器的训练方法、装置、设备及介质
Dai et al. A review of common techniques for visual simultaneous localization and mapping
CN112750164B (zh) 轻量化定位模型的构建方法、定位方法、电子设备
US9852542B1 (en) Methods and apparatus related to georeferenced pose of 3D models
CN111292365B (zh) 生成深度图的方法、装置、电子设备和计算机可读介质
KR20220169472A (ko) 센서 캘리브레이트 방법 및 장치, 전자 기기와 저장 매체
Dong et al. A rgb-d slam algorithm combining orb features and bow
CN114494612A (zh) 构建点云地图的方法、装置和设备
CN112767484B (zh) 定位模型的融合方法、定位方法、电子装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20868024

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20868024

Country of ref document: EP

Kind code of ref document: A1