CN117730239A - Apparatus and method for navigation - Google Patents
Apparatus and method for navigation Download PDFInfo
- Publication number
- CN117730239A CN117730239A CN202180100994.2A CN202180100994A CN117730239A CN 117730239 A CN117730239 A CN 117730239A CN 202180100994 A CN202180100994 A CN 202180100994A CN 117730239 A CN117730239 A CN 117730239A
- Authority
- CN
- China
- Prior art keywords
- sub
- map
- region
- navigation
- transforms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 54
- 230000009466 transformation Effects 0.000 claims abstract description 157
- 238000000844 transformation Methods 0.000 claims abstract description 62
- 238000004422 calculation algorithm Methods 0.000 claims description 32
- 230000008093 supporting effect Effects 0.000 claims description 24
- 230000001131 transforming effect Effects 0.000 claims description 15
- 230000007613 environmental effect Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012876 topography Methods 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- VYMDGNCVAMGZFE-UHFFFAOYSA-N phenylbutazonum Chemical compound O=C1C(CCCC)C(=O)N(C=2C=CC=CC=2)N1C1=CC=CC=C1 VYMDGNCVAMGZFE-UHFFFAOYSA-N 0.000 claims 1
- 238000010276 construction Methods 0.000 description 37
- 238000013507 mapping Methods 0.000 description 17
- 238000013519 translation Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001976 improved effect Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 1
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 1
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 1
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- XTKDAFGWCDAMPY-UHFFFAOYSA-N azaperone Chemical compound C1=CC(F)=CC=C1C(=O)CCCN1CCN(C=2N=CC=CC=2)CC1 XTKDAFGWCDAMPY-UHFFFAOYSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000036316 preload Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012358 sourcing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3863—Structures of map data
- G01C21/387—Organisation of map data, e.g. version management or database structures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3863—Structures of map data
- G01C21/387—Organisation of map data, e.g. version management or database structures
- G01C21/3881—Tile-based structures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3885—Transmission of map data to client devices; Reception of map data by client devices
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Navigation (AREA)
Abstract
A support device is provided for continuous smooth positioning within an area. The support means divides the area into a plurality of sub-areas and generates one or more reference positions within the area. For each reference location, the support means calculates one or more transforms that transform adjacent 2D/3D maps of the sub-region around the reference location to ensure that the adjacent 2D/3D maps are aligned. The one or more transforms are associated with the reference location and may be stored in a database. During navigation, the navigation device determines the most recent reference position from the current position and obtains an associated one or more transformations. The navigation device then dynamically concatenates the maps of the adjacent sub-regions according to the one or more transformations, and provides navigation according to the concatenated maps.
Description
Technical Field
The present invention relates to an apparatus, system and method in the field of navigation technology and computer vision technology. For example, the present invention relates to an apparatus, system and method for supporting in-region navigation.
Background
Accurate and precise map information is often necessary for navigation, especially for autonomous navigation based on computer vision. In order to provide accurate positioning during navigation, map information for navigation systems typically includes a large amount of map construction data such as environmental models, geometric data, and location information. In some conventional approaches, a large scale map may be divided into multiple sub-maps.
Disclosure of Invention
Large scale maps may not adapt well to real environments. For example, small errors may accumulate during the process of mapping according to an iterative algorithm such as the motion estimation structure (structure from motion, sfM or SfM) algorithm or the simultaneous localization and mapping (simultaneous localization and mapping, SLaM or SLaM) algorithm. Thus, drift may occur during navigation. Furthermore, the navigation device may require more time to make pose calculations. Furthermore, maintenance and updating of the large scale map can be complex and time consuming.
When a large scale map or a region of interest corresponding to the large scale map is divided into a plurality of sub-maps of reduced size, the plurality of sub-maps can be simply stored and aligned as needed. One problem with using conventional divided sub-maps is that any misalignment between adjacent sub-maps may cause positioning jumps during navigation. Furthermore, small errors accumulated in the process of making the large scale map may still exist in the plurality of sub-maps.
In view of this, there is a need to address the above-described technical drawbacks of the existing devices to improve in-area navigation.
The apparatus, system and method according to the present invention facilitate providing in-region navigation in a robust, efficient manner.
These and other objects are achieved by the subject matter of the independent claims. Other implementations are apparent in the dependent claims, the description and the drawings.
According to the invention, a local dynamic sub-map stitching method is used to align adjacent sub-maps in accordance with one or more pre-computed and loaded transformations during navigation provided by the navigation device. During navigation, the current location of the navigation device may be used to query one or more pre-computed transformations of the surrounding sub-map. Further, adjacent maps may be stitched together according to the one or more pre-computed transforms to ensure continuous smooth navigation.
A first aspect of the invention provides an apparatus for supporting in-region navigation. The device is used for: one or more reference locations are generated within the region. Further, the device is used for: the region is divided into a plurality of sub-regions. Thus, there is at least one sub-area comprising at least one reference position.
In particular, the navigation may be vision-based navigation, optionally in combination with global navigation satellite system (global navigation satellite system, GNSS) positioning.
Alternatively, the apparatus may be for: a density of the reference locations is determined based on one or more navigation requirements. The density of the reference positions may increase as the need for smooth navigation increases.
Alternatively, each reference location may be a two-dimensional (2D) location or a three-dimensional (3D) location, and may particularly depend on the map type of each sub-area.
For each reference position, the apparatus is for: and acquiring a map of the subarea where the reference position is located and at least one adjacent map of at least one adjacent subarea. The map of the sub-region in which the reference position is located may also be referred to as a current map of a current sub-region. The adjacent sub-region may be a sub-region adjacent to the current sub-region.
In the present invention, the map of the sub-region may be referred to as a sub-map. That is, the current map of the current sub-region may be referred to as a current sub-map, and the adjacent map of the adjacent sub-region may be referred to as an adjacent sub-map. Further, in the present invention, the current sub-region and one of the at least one adjacent sub-region may be collectively referred to as a "set of adjacent sub-regions". Accordingly, the map of the (current) sub-region and the at least one neighboring map of the at least one neighboring sub-region may be collectively referred to as a "set of neighboring maps" or a "set of neighboring sub-maps".
Further, the device is used for: determining one or more transformations; one or more of the current sub-map and the at least one adjacent sub-map are transformed according to the one or more transforms such that the set of adjacent sub-maps are aligned. Alternatively, each transformation may take the form of a vector or matrix, or may include key parameters associated with each transformation. Each transformation may include at least one of: translation transformation; rotating and transforming; and (5) scaling.
The apparatus is then for: the one or more transforms are associated with the reference location. The apparatus may collectively associate the one or more transforms with the reference location.
The apparatus is then for: the associated one or more transforms are provided to a navigation device. In this way, according to the one or more transformations, the navigation service provided by the navigation device can be facilitated, and the map transformation performed by the navigation device can be simplified.
Alternatively, the apparatus may be for: determining the sub-map, the at least one neighboring sub-map and the associated one or more transformations according to pose information provided by the navigation device. Further, the apparatus may determine a most recent reference position relative to the pose information and retrieve the associated one or more transforms. The pose information (or pose for short) may comprise position information and/or orientation information. Further, the location information may include 2D or 3D coordinates. The 2D or 3D coordinates may be global coordinates, for example, GNSS coordinates. Alternatively, the 2D or 3D coordinates may be local coordinates, for example, relative coordinates with respect to a reference position of the region of interest. The direction information may include tilt, pitch, and yaw information in 3D space, and may also include angles in 2D space.
In this way, each reference position may be used as an index during navigation to determine the associated transformation or transformations. The navigation device may then be used to: and according to the one or more transformations, correspondingly combining the group of adjacent sub-maps to acquire a local alignment map. The local alignment map may ensure continuous smooth navigation. According to the local alignment map, bouncing and drifting can be avoided during the navigation. In addition, navigation accuracy can be improved.
Alternatively, the map of a particular sub-region may be reused for different stitched scenes, depending on the different reference locations. Maintenance and updating of the maps of the plurality of sub-areas may be simplified.
Alternatively, during navigation, alignment between multiple sets of different adjacent maps may be achieved and dynamically updated. Thus, there is no need to calculate a global alignment of all of the maps of the plurality of sub-regions.
In the present invention, the device for supporting navigation is also referred to as a "supporting device".
In one implementation of the first aspect, the sub-region and the at least one adjacent sub-region may share an overlapping portion. In this case, in order to transform one or more of the map and the at least one neighboring map according to the one or more transforms, the apparatus may be specifically configured to: one or more maps of the sub-region and the at least one adjacent sub-region are transformed such that the maps are aligned at least in the overlapping portion.
Alternatively, in case the sub-region and the at least one adjacent sub-region do not share an overlapping portion but share at least one common edge, the apparatus may align the map of the sub-region and the at least one adjacent sub-region by matching edge features of the map.
In an implementation manner of the first aspect, the apparatus may be configured to: determining the shape and/or size of each sub-region of the plurality of sub-regions according to one or more of the following criteria:
-the topography of the area;
-a geographical density of the environmental data of the area;
-available computing resources of the apparatus for processing the environment data associated with each sub-area.
The divided sub-regions may have different shapes and sizes. That is, each sub-region may have a separate shape and size.
Further, the topography of the area may include one or more of the following: terrain information; road topology; road name; point of interest (point of interest, POI) boundaries.
In this way, each divided sub-region can be determined according to the actual application scenario of navigation and device capabilities. Therefore, navigation flexibility can be improved.
In an implementation manner of the first aspect, to obtain the map of each sub-area, the apparatus may be configured to: acquiring original data; and generating a representation of each sub-area as the map according to the acquired original data. The raw data may include one or more of the following information:
-one or more images;
-light detection and ranging (light detection and ranging, liDAR) data;
-coordinate information of one or more GNSS;
-geometric data;
-road data;
optionally, the one or more images may include one or more of: a color image; a monochrome image; and (5) a depth image.
Further, the map of each sub-region may be a superposition of the original data.
In one implementation of the first aspect, the map of each sub-region may further comprise at least one or more representations of:
-a 2D model of the sub-region;
-a 3D model of the sub-region.
The 2D model may be a 2D point cloud representing the sub-region or a 2D grid representing the sub-region, and the 3D model may be a 3D point cloud representing the sub-region or a 3D grid representing the sub-region.
Alternatively, the 2D model or the 3D model may represent environmental data of each respective sub-region, such as, but not limited to, road, building, POI, river, lake, mountain, etc. terrain.
In one implementation of the first aspect, the map of each sub-region may further include pose information associated with the one or more representations.
Alternatively, the pose information may indicate a position and orientation associated with each representation. The pose information may include a pose of a sensor for collecting the raw data. Alternatively or additionally, the pose information may include a pose used by a map construction platform to construct a map.
In an implementation manner of the first aspect, in a case that the map and the at least one neighboring map share the overlapping portion, the apparatus may be specifically configured to: and determining the relative pose between the subarea and the map of the at least one adjacent subarea according to the overlapped part. The apparatus may then transform the map of the sub-region or the at least one sub-region according to the relative pose.
The relative pose may also be determined by using an iterative closest point (iterative closest point, ICP) algorithm from the representations of the set of adjacent sub-regions. The representation may be a 2D/3D model or map.
Optionally, the relative pose may be included in the one or more transformations. The relative pose may be relative to the pose information included in the current sub-map.
In one implementation of the first aspect, to generate the one or more reference positions, the apparatus may generate the one or more reference positions at one or more landmarks within the region.
The one or more landmarks may include one or more of roads and POIs.
In an implementation manner of the first aspect, to generate the one or more reference positions, the apparatus may be configured to: the one or more reference locations are generated within one or more accessible regions of the region.
In this way, the one or more reference locations may be efficiently allocated and computing resources may be saved.
In an implementation manner of the first aspect, to generate the one or more reference positions, the apparatus may be configured to: the one or more reference locations are generated uniformly within the region. Alternatively, the apparatus may randomly generate the one or more reference locations within the region. In either way, the generation of the reference position can be simplified, and the time consumption for preparing the reference position can be reduced.
In an implementation manner of the first aspect, the apparatus may be configured to: the number and/or distribution of the one or more reference locations is determined based on the complexity of the map for each sub-region.
For denser subregions, the density of the reference locations may be higher.
In one implementation manner of the first aspect, to determine the one or more transforms, the apparatus may be specifically configured to:
-calculating a local alignment transformation centred on the reference position between the sub-area and the map of the at least one adjacent sub-area using a stitching algorithm;
-obtaining an output of the stitching algorithm as one of the one or more transformations.
Alternatively, the stitching algorithm may be used to match edge features between the set of adjacent sub-regions, or to match overlapping portions between the set of adjacent sub-regions.
In one implementation of the first aspect, each transformation may be a 2 degree of freedom (degree of freedom, doF) or a 3DoF or 6DoF or 7DoF transformation, wherein the 2DoF transformation may involve a two-axis translational transformation, the 3DoF transformation may involve a three-axis translational transformation or the 2DoF transformation and direction, the 6DoF transformation may involve three-axis rotational and translational transformations, and the 7DoF transformation may involve three-axis rotational and translational transformations and scaling.
In an implementation manner of the first aspect, in order to associate the one or more transforms with the at least one reference position included in each sub-region, the apparatus may be configured to: the one or more transformations of the sub-region are stored in a database indexed according to each of the at least one reference location.
Alternatively, the support means may provide the database directly to the navigation device before navigation begins, in order to provide the associated one or more transformations to the navigation device. In this case, the navigation device may retrieve the one or more transformations by querying the database according to current pose information during navigation.
In an implementation form of the first aspect, to provide the one or more transforms to the navigation device, the device may be configured to:
-receiving pose information from the navigation device;
-determining a reference position from the pose information;
-determining said one or more transformations from said determined reference position.
In order to determine the reference position from the pose information, the apparatus may select a reference position nearest to the position included in the received pose information as the determined reference position.
A second aspect of the present invention provides a navigation apparatus for providing a navigation service to devices within an area. The navigation device is used for:
acquiring the current pose of the equipment;
acquiring a map of a current subarea and at least one adjacent subarea according to the current pose;
acquiring one or more current transformations from a support device for supporting the navigation service according to the current pose;
transforming the map according to the one or more current transforms;
and providing the navigation service according to the transformed map.
For example, the support means for supporting the navigation service is the means for supporting navigation according to the first aspect. Supporting navigation services is the same as supporting navigation according to the present invention.
Optionally, the current pose may include position information and/or direction information. The location information may include 2D or 3D coordinates. The direction information may include tilt, pitch, and yaw information in 3D space, and may also include angles in 2D space.
Alternatively, the navigation device may acquire a map of all the sub-regions within the region including the current sub-region and the at least one adjacent sub-region from a map construction platform before navigation, and may store the acquired sub-map locally.
Alternatively, the navigation device may acquire all the sub-maps from the support device, in particular if all the sub-maps are generated by the support device. In this case, the navigation apparatus may receive all of the sub-maps from the support apparatus before navigation, and may store the received sub-maps locally.
Alternatively, the navigation device may acquire the maps of the current sub-region and the adjacent sub-region from the map construction platform or the support device during the navigation. In this case, the navigation device may transmit the current pose to the map construction platform or the support device. In this way, the storage requirements of the navigation device can be reduced.
Alternatively, the navigation device may obtain a database comprising the one or more current transformations from the support device before starting the navigation service. In this way, the navigation device may be operated in an offline mode in which communication between the support device and the navigation device during navigation may not be required. Communication delay can be avoided. In the offline mode, to obtain the one or more current transformations, the navigation device may be configured to:
Selecting the nearest reference position from the database relative to a current position included in the current pose;
the one or more current transforms associated with the most recent reference position are retrieved from the database.
Alternatively, the navigation device may acquire the one or more current transformations during navigation. In this case, the navigation device may provide the current pose to the support device during navigation. In this way, the navigation device can be operated in an online mode, and the storage requirements for the navigation device can be reduced. In the online mode, to obtain the one or more current transformations, the navigation device may be to:
transmitting the current pose to the supporting device;
the one or more current transformations are received from the support apparatus.
It should be noted that the one or more current transformations may be considered to be acquired from the support means, both in the offline mode and in the online mode.
In order to transform the map according to the one or more current transforms, the navigation device may be configured to not transform a particular one of the maps if there is no transform corresponding to the particular map. Alternatively, the navigation device may be configured to: combining the specific map with the transformed neighboring map; and providing the navigation service according to the combined map.
Alternatively, the device providing the navigation service may be a handheld device (e.g., a terminal, a cell phone, a tablet, a GNSS station, etc.), an augmented reality (augmented reality, AR) or Virtual Reality (VR) device, a robot, a land vehicle (e.g., a car, a bus, a truck, a tram, a subway, a train), a boat, an airplane (e.g., an unmanned aerial vehicle), etc.
In an implementation manner of the second aspect, the navigation device may be further configured to:
updating the current pose;
updating the map and the one or more transforms according to the updated pose;
transforming the updated map according to the one or more updated transforms;
and continuing to provide the navigation service according to the updated and transformed map.
Alternatively, the navigation device may be configured to: the current pose of the device is continuously updated during the navigation service.
A third aspect of the invention provides a system for in-region navigation. The system comprises a support device according to the first aspect or any implementation thereof and a navigation device according to the second aspect or any implementation thereof.
Alternatively, the support means may be a device physically separate from the navigation means. Alternatively, the support means may be a device attachable to the navigation device. Alternatively, the support device may be a model inside the navigation device.
A fourth aspect of the invention provides a method for supporting intra-area navigation. The method is performed by a support device and comprises the steps of:
generating one or more reference locations within the region;
the region is divided into a plurality of sub-regions.
For each reference position, the method further comprises:
acquiring a map of a subarea where the reference position is located and an adjacent map of at least one adjacent subarea;
determining one or more transformations;
transforming one or more of the map and the at least one neighboring map according to the one or more transforms such that the map and the at least one neighboring map are aligned;
associating the one or more transforms with the reference location;
the one or more transforms are provided to a navigation device.
In an implementation manner of the fourth aspect, the sub-region and the at least one adjacent sub-region may share an overlapping portion.
In an implementation manner of the fourth aspect, the method may further include: the shape and/or size of each sub-region is determined according to one or more of the following criteria:
topography of the area;
geographic density of environmental data of the region;
the available computing resources of the apparatus are used to process the environmental data associated with each sub-region.
In an implementation manner of the fourth aspect, acquiring the map of each sub-area may include:
obtaining raw data, the raw data comprising at least one of:
one or more images;
LiDAR data;
coordinate information of one or more global navigation satellite systems;
geometric data;
road data;
and generating a representation of each sub-area as the map according to the acquired original data.
In one implementation of the fourth aspect, the map of each sub-region may comprise at least one or more representations of:
a two-dimensional (2D) model of the sub-region;
a three-dimensional (3D) model of the sub-region.
The 2D model may be a 2D point cloud representing the sub-region or a 2D grid representing the sub-region, and the 3D model may be a 3D point cloud representing the sub-region or a 3D grid representing the sub-region.
In one implementation of the fourth aspect, the map of each sub-region further comprises pose information associated with the one or more representations.
In one implementation of the fourth aspect, determining the one or more transforms may include:
a relative pose between the map and the at least one neighboring map is determined from the overlapping portion, wherein the determined relative pose is one of the one or more transformations.
In an implementation manner of the fourth aspect, generating the one or more reference positions may include: the one or more reference locations are generated at one or more landmarks within the region.
In an implementation manner of the fourth aspect, generating the one or more reference positions may include: the one or more reference locations are generated within one or more accessible regions of the region.
In an implementation manner of the fourth aspect, generating the one or more reference positions may include:
generating the one or more reference locations uniformly within the region; or (b)
The one or more reference locations are randomly generated within the region.
In an implementation manner of the fourth aspect, the method may further include: the number and/or distribution of the one or more reference locations is determined based on the complexity of the map for each sub-region.
In one implementation of the fourth aspect, determining the one or more transforms may include:
calculating local alignment transformation centering on the reference position between the map and the at least one adjacent map by adopting a stitching algorithm;
an output of the stitching algorithm is obtained as one of the one or more transforms.
In one implementation of the fourth aspect, each transformation may be a 2DoF or 3DoF or 6DoF or 7DoF transformation, wherein the 2DoF transformation may involve a two-axis translational transformation, the 3DoF transformation may involve a three-axis translational transformation or the 2DoF transformation and direction, the 6DoF transformation may involve three-axis rotational and translational transformations, and the 7DoF transformation may involve three-axis rotational and translational transformations and scaling.
In one implementation of the fourth aspect, associating the one or more transforms with the reference location may include: the one or more transforms are stored in a database indexed according to the reference location.
In one implementation of the fourth aspect, providing the one or more transforms to the navigation device may include:
receiving pose information from the navigation device;
determining a reference position according to the pose information;
the one or more transforms are determined based on the determined reference locations.
A fifth aspect of the invention provides a method for providing navigation services to devices within an area. The method is performed by a navigation device and comprises the steps of:
acquiring the current pose of the equipment;
acquiring a map of a current subarea and at least one adjacent subarea according to the current pose;
acquiring one or more current transformations from a support device for supporting the navigation service according to the current pose;
transforming the map according to the one or more current transforms;
and providing the navigation service according to the transformed map.
In an implementation manner of the fifth aspect, the method may further include:
updating the current pose;
updating the map and the one or more transforms according to the updated pose;
transforming the updated map according to the one or more updated transforms;
And providing the navigation service according to the updated and transformed map.
A sixth aspect of the invention provides a computer program product comprising: program code for performing, when executed on a computer, a method according to the fourth aspect or any implementation thereof.
A seventh aspect of the invention provides a computer program product comprising: program code for performing, when executed on a computer, a method according to the fifth aspect or any implementation thereof.
An eighth aspect of the invention provides a computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to the fourth aspect or any implementation thereof.
A ninth aspect of the invention provides a computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to the fifth aspect or any implementation thereof.
A tenth aspect of the invention provides a chipset comprising instructions that, when executed by the chipset, cause the chipset to perform the method according to the fourth aspect or any implementation thereof.
An eleventh aspect of the invention provides a chipset comprising instructions that, when executed by the chipset, cause the chipset to perform the method according to the fifth aspect or any implementation thereof.
It should be noted that all the apparatuses, devices, elements, units and methods described in the present invention may be implemented in software or hardware elements or any combination thereof. The steps performed by the various entities described in this application, as well as the functions to be performed by the various entities described are all intended to mean that the various entities are adapted to, or are used to, perform the various steps and functions. Even though in the description of specific embodiments below the specific functions or steps to be performed by external entities are not reflected in the description of specific detailed elements of the entity performing the specific steps or functions, it should be clear to a person skilled in the art that these methods and functions may be implemented in respective software or hardware elements, or any combination of such elements.
Drawings
The aspects and implementations described above will be described in the following description of specific embodiments with reference to the accompanying drawings, in which:
FIG. 1A shows one example of two sub-maps;
FIG. 1B shows another example of two sub-maps;
FIG. 2 shows a schematic of a method;
FIG. 3 illustrates a navigation system;
FIG. 4 shows one illustrative example of region partitioning;
FIG. 5 shows one illustrative example of a sub-region;
FIG. 6 shows one illustrative example of region division and reference location generation;
FIG. 7 shows one illustrative example of map stitching and transformation computation for a reference location;
FIG. 8 shows one illustrative example of region partitioning;
FIG. 9 shows a schematic of a method;
fig. 10 shows a schematic of another method.
Detailed Description
In fig. 1 to 10, the corresponding elements may have the same features and may function in the same manner.
Fig. 1A and 1B show examples of two sub-maps. The device for supporting in-area navigation (also simply referred to as "supporting device" in the present invention) is configured to: dividing the region into a plurality of sub-regions; one or more reference locations are generated within the region. It should be noted that in the present invention, this region may be referred to as a region of interest that provides or supports navigation. The means for supporting navigation may be referred to as supporting means.
As exemplarily shown in fig. 1A and 1B, two sub-maps (101, 103) correspond to two divided sub-areas. Each sub-region may represent a portion of the region. In the present invention, each sub map and each sub area may be in one-to-one correspondence. That is, the sub-regions may be represented by corresponding sub-maps. Thus, for convenience, sub-regions and corresponding sub-maps may have the same reference numerals. For example, in the present invention, the sub-area 103 and the sub-map 103 may be used simultaneously. Further, the term "sub-map" may be used to refer to a corresponding "sub-region," and the sub-map may also be referred to as a map.
Alternatively, the area may be an outdoor area such as, but not limited to, a park, a region, and a city. Alternatively, the area may be an indoor area such as, but not limited to, a shopping mall, warehouse, and office building. Alternatively, the area may be a combination of an outdoor area and an indoor area, such as, but not limited to, an industrial park including roads, office buildings, factories, and warehouses.
Although the sub-maps (101, 103) are shown as 2D maps in fig. 1A and 1B, it should be noted that the sub-maps may include 2D and/or 3D maps. Optionally, the sub-map may also include map construction data that may be superimposed on the 2D/3D map. The map build data may include images, liDAR scans, POI or GNSS coordinates, or any combination thereof.
The two sub-regions (101, 103) may share a common boundary and may not share overlapping portions, as exemplarily shown in fig. 1A. In other cases, as exemplarily shown in fig. 1B, two sub-regions (101, 103) may share the overlapping portion 102. That is, in this case, the overlapping portion 102 is a part of the sub-region 101 and a part of the sub-region 103. In both cases, sub-region 103 may be referred to as an adjacent sub-region of sub-region 101, while sub-region 101 may also be referred to as an adjacent sub-region of sub-region 103. The two sub-regions (101, 103) may be referred to as a pair of adjacent sub-regions. If there are more than two adjacent regions, a set of adjacent sub-regions may be used. Alternatively, adjacent sub-regions of a particular sub-region may be referred to as sub-regions adjacent to the particular sub-region, the sub-regions sharing a common boundary or overlap with the particular sub-region. If there are three or more sub-regions, any two sub-regions sharing a common boundary or overlap are also referred to as a pair of adjacent sub-regions. That is, a set of adjacent sub-regions may include one or more pairs of adjacent sub-regions. Alternatively, a pair of adjacent sub-regions may be the minimum transform calculation unit.
Further, reference positions are exemplarily shown in fig. 1A and 1B. The support means is for determining one or more transformations and transforming a set of adjacent maps such that the adjacent maps are aligned. Alternatively, the alignment may point to each pair of adjacent sub-regions comprised in a set of adjacent sub-regions. Alternatively, each transformation may include at least one of a translation transformation (or simple translation), a rotation transformation (or simple rotation), and a scaling transformation (or simple scaling). Translation may be referred to as a geometric transformation that moves each point of a graph or space the same distance in a given direction. The rotation may be referred to as a geometric transformation that moves each point of the graph or space by the same angle. Scaling may be a linear transformation that expands (increases) or contracts (decreases) the coordinates by a scaling factor. The scaling may be an isotropic scaling in which the scaling factor for all coordinates has the same value.
For example, the transformation of the 2D map construction data may be applied by using the following matrix operation:
where (x, y) represents the 2D coordinates before transformation, (x ', y') represents the 2D coordinates after transformation, (h, k) represents translation, a represents rotation, and m represents scaling. In this example, each reference position may be associated with a separate combination of (h, k), a, and m.
It should be noted that the matrix given above is merely an example for illustrating the transformation. There are other matrices in the art that represent translation, rotation, and scaling. Alternatively, it is sufficient that the transformation may comprise information indicative of at least one of translation, rotation and scaling.
In fig. 1A and 1B, the support means may determine that the sub-map 103 may be moved up a certain distance D for the reference position, so that the two sub-maps (101, 103) may be aligned along a common boundary or at overlapping portions. Thus, the transformation of the sub-map 103 may be performed by applying the following matrix:
in this example, no rotation or scaling is required. Thus, a=0 and m=1. The transform (e.g., information representing (h, k) = (0, d), a=0, and m=1) may then be associated with the reference location. Further, since the sub map 101 does not need to be transformed, the transformation of the sub map 101 may be null or invalid. It should be noted that the null value of the scaling factor should be understood as 1. That is, if the scaling factor is 1 or is undefined in the transformation, no scaling is performed.
Determining a transformation and transforming a set of adjacent maps may be collectively referred to as map stitching and transformation computation. Algorithms for aligning and stitching 2D/3D/depth images corresponding to adjacent maps are well known in the art. For example, a factor graph model may be used, and nonlinear optimization may be involved. Alternatively, the support apparatus may use the Levenberg-Marquardt algorithm and Gaussian-Newton iterations during map stitching and transformation computation. Details about the algorithm should be well known to those skilled in the art and will not be discussed in detail herein.
Alternatively, the support means may be arranged to store one or more transformations indexed according to the reference location in the database. The database may be stored in a storage medium of the support apparatus. Alternatively, the support apparatus may determine a unique Identification (ID) of each reference position. The unique ID may be a serial number or GNSS coordinates of each reference position, or may be a hash value of the GNSS coordinates.
Fig. 2 shows a schematic diagram of a method performed by a support device. The support means are for performing region division and reference position generation 201. It should be noted that there is no particular order of execution between the region division and the reference position generation.
In the region division and reference position generation 201, the support means may be used to divide the region into a plurality of sub-regions. The coverage of all sub-regions may be the same as the coverage of the region of interest. Some sub-regions may share overlapping portions.
Alternatively, the support means may be adapted to generate map construction data from the raw data of the area. Raw data may be collected from different sensors and may include one or more of the following: one or more images of the area, liDAR data, GNSS coordinates, geometry data, POIs, and road information. Map construction data may be generated using some SLAM algorithms known in the art. In the present invention, map construction data of a sub-region may be simply referred to as a map.
Alternatively, the region of interest may already include mapping data provided by a mapping platform or by crowd sourcing means. Further, the map construction data may be constructed by a map construction platform according to map construction algorithms (such as, but not limited to, SLAM and SFM algorithms) well known in the art. The map construction data may comprise a 2D model or a 3D model of the or each sub-region of interest, which may be referred to as a 2D map or a 3D map. Optionally, the map construction data may further comprise one or more of the following information of the region or sub-region: environmental images, liDAR scans, POI information, terrain of an area, road information, and GNSS coordinates. If an area is divided into a plurality of sub-areas, each sub-area may include corresponding map data inherited from the map data of the area. The support means may be adapted to store the map construction data for each sub-area and to provide the map construction data to the navigation means during navigation. Alternatively, the map construction data of the sub-region may also be provided by the map construction platform to the navigation device.
Alternatively, the support means may employ a partitioning algorithm to partition the region of interest. For example, the support device may adjust the size and shape of each individual sub-region according to one or more of road topology, road semantics, and geographic density of the map build data. Alternatively, the support device may divide the areas such that the calculation time of the map stitching of each divided sub-area is comparable.
Further, the support apparatus generates one or more reference locations within the region of interest. Alternatively, two or more reference positions may be generated according to regular sampling of the region of interest, wherein the reference positions may be uniformly located in the region of interest. Alternatively, two or more reference positions may be generated from irregular sampling. Alternatively, the one or more reference locations may be located only within an accessible region of the region of interest.
Alternatively, the support device may determine the density and number of one or more reference locations based on the navigation requirements. Denser reference locations may enable smoother navigation, but may also result in a larger database for storing transformations.
After the region of interest is divided, the one or more reference locations are also divided into sub-regions, respectively.
The support means is then used to perform map stitching and computation 202 for each reference location. For convenience, the sub-region in which the reference position is located should be referred to as the current sub-region. Optionally, for each reference position, the support means may determine one or more adjacent sub-regions relative to the current sub-region. Alternatively, the support means may determine the distance and may exclude one or more adjacent sub-areas that are not within the distance range from the current reference position. In this way, one or more further adjacent sub-regions of the reference location may be excluded and computational resources may be reduced.
Alternatively, the support apparatus may be used to perform map stitching and transformation calculations 202 using conventional image alignment and stitching algorithms. Alternatively, the stitching algorithm may be based on a factor graph model. Further, non-linear optimization may be used.
In some embodiments of the invention, one or more of global and relative constraints may be applied to conventional image alignment and stitching algorithms. Global constraints, which may also be referred to as absolute constraints, may be used to ensure that certain map-building data in the transformed sub-map corresponds to static values, such as GNSS coordinates, carefully chosen pivot points, semantic features with known GNSS coordinates, and so on. The relative constraints may be used to ensure that the map construction data of adjacent sub-maps are aligned between common edges or at overlapping portions (if available). Further, if the map construction data of a pair of adjacent sub-maps includes 2D or 3D point clouds and associated pose information, the support apparatus may calculate the relative pose between the pair of adjacent sub-maps using an ICP algorithm. The relative pose may include relative translational and rotational transformations that may be used to achieve optimal alignment between two adjacent sub-maps. The relative pose may also include a scaling factor. By applying global constraints and relative constraints, the stitched sub-map may be accurate within a global scope due to global constraints and have continuity across the sub-map due to relative constraints. In the process of local map stitching, small errors introduced in the process of collecting and making map construction data can be reduced for adjacent sub-maps.
There may be one transformation for each sub-map for each reference location. If a sub-map does not require a transformation, the sub-map may not include a transformation. When using a relative transformation, there may be one relative transformation for each pair of adjacent sub-maps.
Each transformation may be based on a 3DoF or 6DoF transformation. Optionally, each transformation may additionally include a scaling factor as an additional DoF. Alternatively, the 3DoF transform may correspond to a 2D map, and may include a 2DoF translational transform and a 1DoF rotational transform in 2D space. If no rotation transformation is present, the 3DoF transformation may also take the form of a 2DoF, which may include a 2DoF translation transformation in 2D space. The 6DoF transform may correspond to a 3D map and may include a 3DoF translational transform and a 3DoF rotational transform in 3D space. If no rotation transformation is present, the 6DoF transformation may also take the form of a 3DoF, which may include only a 3DoF translation transformation in 3D space.
The support means is further for associating one or more transformations with the reference location. Alternatively, the support means may store one or more transforms in a transform database indexed according to the corresponding reference location. Alternatively, each reference location may be transformed into a unique value or ID, such that the associated transformation or transformations may be easily indexed by the unique value or ID. For example, the support apparatus may calculate a hash value of the GNSS coordinates of each reference position. Alternatively, the transformation database may take the form of a look-up table with O (1) access complexity and high memory footprint. Alternatively, the transformation database may take the structure of a KD tree with O (n log (n)) access complexity and moderate memory footprint. Alternatively, the support device may use any other suitable database in the art.
The support means may be adapted to perform steps 201 and 202 only once as a navigation preparation. Optionally, when updating the map-building data is required, the support means may be adapted to update the map-building data of the one or more affected sub-areas and to recalculate the corresponding one or more transformations of any reference locations comprised within the one or more affected sub-areas and within adjacent sub-areas of the one or more affected sub-areas. In this way, maintenance and updating of the map of the region of interest may be simplified.
The support means may then be adapted to provide the stored map data and transformation database of the sub-areas to the navigation means for providing navigation services to the devices within the area of interest. Alternatively, the map-building data of the sub-region may be provided by a map-building platform. This is not limited to the embodiments of the present invention. It should be noted that the navigation device may be an internal unit included in the apparatus or may be an external unit attachable to the apparatus. This is not limited to the embodiments of the present invention. Alternatively, the navigation means may be used to provide map-based positioning to the device. Additionally, the navigation device may be assisted by a GNSS positioning module.
During navigation, the navigation device may perform dynamic map alignment 203 and positioning 204. And acquiring the current pose. The pose may include location information and optionally direction information of the device. The location information may be 2DoF or 3DoF, for example, 2D coordinates or 3D coordinates. The position information included in the current pose may be used to determine a most recent reference position such that the associated one or more transforms may be retrieved from a transform database according to the determined reference position. Alternatively, the current pose may be a positioning result obtained from the most recent positioning 204. Alternatively, for example, if no previous positioning results are provided, the current pose may be acquired by one or more sensors of the device (e.g., GNSS module, camera, radar, liDAR sensor, inertial measurement unit, etc.).
In some embodiments of the invention, the support device may provide sub-maps and transformation databases for all sub-regions within the region of interest to the navigation device prior to navigation. That is, the navigation device may preload all sub-maps and the transformation database. In this case, a communication link between the navigation device and the support device is not required during navigation, and the navigation device can provide navigation services in an offline mode. In this case, the navigation device may be configured not to provide the current pose to the support device. The reference position may be determined locally on the navigation device.
In some other embodiments of the invention, the navigation device may be used to provide the current pose to the support device so that the support device may determine the reference position. In this case, a communication link between the navigation device and the support device is required during navigation, and the navigation device can provide navigation services in an online mode. The navigation device may then obtain a map around the reference location and associated one or more transformations from the support device. Alternatively, the navigation device may obtain the map from the map construction platform.
However, in all embodiments of the invention, the navigation device obtains one or more transformations at least from the support device. It should be noted that this also includes: the navigation device is used in the case of retrieving a transformation database from the support device, because one or more transformations are initially provided by the support device in an online mode and an offline mode.
The navigation device is then used to perform dynamic map alignment 203. To perform the dynamic map alignment 203, the navigation device is used to transform one or more sub-maps of the corresponding sub-region according to one or more transforms in the transform database indexed according to the nearest reference position. Thus, adjacent sub-maps may be aligned along a common boundary or overlapping portion. The navigation device may then be used to perform positioning 204 from the aligned adjacent sub-maps to obtain the latest accurate pose of the apparatus. The positioning may also be referred to as pose estimation. For example, the positioning may be map-based positioning. Thus, the navigation device can provide continuous smooth navigation services according to the transformed sub-map, which is globally accurate and locally consistent. The detailed information about the location 204 should be well known to those skilled in the art and will not be discussed in detail herein.
During navigation, the navigation device may be used to obtain updated poses as the device moves. The updated pose may cause the reference position to change. Thus, the updated pose may be used to obtain one or more updated sub-maps and one or more updated transformations. The navigation device may then transform the one or more updated sub-maps according to the one or more updated transforms and continue to provide navigation services. It should be noted that one or more updated transforms may indicate a transformation between a previously acquired sub-map (e.g., a sub-map acquired prior to updating) and at least one updated neighboring sub-map. For example, the navigation device may previously provide navigation within sub-region 1 and sub-region 2. After pose update, the navigation device may provide navigation within sub-region 2 and sub-region 3. In this case only a map of the sub-area 3 needs to be acquired. Thus, the navigation device may acquire only one updated sub-map.
It should be noted that in order to perform positioning 204, the navigation apparatus may calculate the pose of the device using positioning algorithms well known in the art. For example, the positioning algorithm may be, but is not limited to, the following:
SLaM algorithm in positioning mode;
a learned pose regressor;
an ICP-like alignment algorithm;
a Perspective-like n-Point (PnP) element Point alignment algorithm.
Fig. 3 shows a navigation system. The navigation system comprises the support device 310 described with reference to fig. 1 and 2 and the navigation device 320 described with reference to fig. 2. Fig. 3 also shows an optional mapping unit or platform 330 that may be used to provide mapping data 311 to support device 310 and/or navigation device 320. The map construction unit 330 may not be required if the support apparatus 310 can generate the map construction data 311 by itself. Alternatively, the mapping platform 330 may be integrated in the support device 310. Further, the mapping unit 330 may be used to instruct a single mapping device to scan the region of interest to collect the mapping data 311. Alternatively, the mapping unit 330 may be used to direct a set of distributed collection devices that may be used to collect the mapping data 311 of the region of interest in a crowdsourcing manner.
The support means 310 is for performing region division and reference position generation 201 to obtain a plurality of sub-regions and one or more reference positions 312. Alternatively, the support apparatus 310 may inform the map construction platform 330 of the divided sub-areas, so that the map construction platform may collect map construction data 311 of the sub-areas and provide it to the support apparatus 310 and/or the navigation apparatus 320.
The support apparatus 310 then performs map stitching and transformation computation 202 to obtain one or more transformations 313 for each reference location. One or more transforms 313 acquired are then associated with each reference location and may be stored in a transform database 314.
During navigation, the navigation device 320 may obtain a current sub-map of the current sub-region, one or more adjacent sub-maps of one or more adjacent sub-regions, and corresponding one or more transformations according to the current pose.
Alternatively, the navigation device 320 may acquire all the sub-maps 321 from the map construction unit 330 or the support device 310 before navigation. Further, the navigation device 320 may obtain the transformation database 313 from the support device 310.
Alternatively, the navigation device 320 may also obtain a map of the current sub-area and at least one neighboring sub-area from the map construction platform 330 or the support device 310 during navigation. In this case, the navigation device 320 may provide the current pose to the map construction platform 330 or the support device 310, not shown in fig. 3.
The navigation device performs the positioning 204 and outputs the current pose 322 of the device. The current pose 322 may be used to determine the nearest reference position for the next round of positioning 204. Based on the determined reference position, the navigation device 320 obtains the associated one or more transforms. Then, the navigation device 320 performs map alignment 203 to acquire a locally aligned adjacent sub-map 323. The navigation device 320 then provides navigation services according to the locally aligned adjacent sub-map 323. During navigation, the current pose 322 may be updated as the device moves. The navigation device 320 may then continually update one or more sub-maps and one or more transformations according to the latest pose 322 and perform map alignment 203. Thus, the map alignment 203 may also be referred to as a dynamic map alignment 203.
Fig. 4-8 show illustrative examples based on a common area 400.
Fig. 4 shows one illustrative example of region division. The region of interest 400 is divided into four sub-regions 401-404, also labeled as sub-regions I-IV in fig. 4.
Fig. 5 shows an illustrative example of sub-regions 401-404. In this example, each two sub-regions in fig. 5 may be referred to as a pair of adjacent sub-regions. That is, for sub-region 401, each of sub-region 402, sub-region 403, and sub-region 404 is an adjacent sub-region. Alternatively, for sub-region 401, there is a set of three adjacent sub-regions 402, 403 and 404. Alternatively, sub-regions 401, 302, 403, and 404 may be collectively referred to as a set of adjacent sub-regions. Further, in this example, each two sub-regions share the overlapping portion marked in fig. 5.
Fig. 6 shows one illustrative example of region division and reference position generation 201. In this example, a plurality of reference locations are generated along a road within the region of interest 400. Further, multiple reference locations may be "inherited" in sub-maps 401-404, respectively.
FIG. 7 shows one illustrative example of map stitching and transformation computation 202 for a reference location. In particular, FIG. 7 shows map stitching and transformation computation 202 for a particular reference location marked in region 701. The region 701 corresponds to the region of interest 400 and is transformed according to four transforms associated with that particular reference position. It should be noted that for this particular reference position, the transformation of sub-region I and sub-region II may be globally constrained, while the transformation of sub-region III and sub-region IV may be relatively constrained, since the particular reference position is closer to sub-region III and sub-region IV than to sub-region I and sub-region II. More specifically, in this illustrative example, the particular reference position is located even in the overlapping portion of sub-region III and sub-region IV.
In some embodiments of the invention, the predetermined distance may be defined by the support means. Sub-maps that are not within a range of distances relative to a reference location may be transformed to be subject to global constraints, while sub-maps within a range of distances may be transformed to be subject to relative constraints.
It should be noted that similar steps described in connection with fig. 7 may be repeated for other reference positions.
Some examples of one or more transformations and associated reference locations for the region of interest 400 are shown in table 1. Specifically, table 1 shows: for reference position 1, only the map of sub-region I needs to apply transform 1. Thus, the transform domains of other sub-regions may be invalid. For reference position 2, three transforms 2-4 are applied to sub-region I through sub-region III, respectively. For reference position 3 three transformations 5-7 are applied to sub-region I, sub-region II and sub-region IV, respectively.
Some examples of determining constraints applied by the transformation are shown in table 2. For example, as shown in Table 2, transforms 2-4 applied to sub-regions I through III are obtained by applying global constraints 1-3 to sub-regions I through III, and applying relative constraints 1-2 to each of sub-regions I-II and sub-regions II-III. The objective function according to global and relative constraints may be as follows:
Wherein,is an optimized transformation of sub-region i,/->Is a transformation obtained according to the global constraint of sub-region i, < - > j->Is a relative transformation obtained from the relative constraint between sub-regions i and j.
TABLE 1-one or more transforms associated with each reference position
TABLE 2 one or more constraints that determine one or more transforms
/>
Fig. 8 shows one illustrative example of positioning 204 and dynamic map alignment 203.
Alternatively, after acquiring the current pose, the navigation device may determine a reference position having the shortest distance to the current pose, and may retrieve one or more current transformations according to the determined reference position. In this case, the transformation database may be preloaded into the navigation device prior to navigation.
Alternatively, the navigation device may also provide the current pose to the support device. The support device may then determine one or more current transforms based on the current pose. In this case, the transformation database may not be preloaded.
As shown in fig. 8, if the pose of the device is changing during navigation, the positioning 204 and dynamic map alignment 203 may be performed continuously.
Fig. 9 shows a schematic diagram of a method 900.
The method 900 is performed by a support apparatus for supporting navigation within an area and comprises the steps of:
Step 901: generating one or more reference locations within the region;
step 902: the region is divided into a plurality of sub-regions.
For each reference position, the method further comprises the steps of:
step 903: acquiring a map of a subarea where a reference position is located and at least one adjacent map of at least one adjacent subarea;
step 904: determining one or more transformations;
step 905: transforming one or more of the map and the at least one neighboring map according to the one or more transforms such that the map and the at least one neighboring map are aligned;
step 906: associating one or more transforms with the reference location;
step 907: one or more transforms are provided to the navigation device.
It should be noted that there is no strict order of execution between step 901 and step 902. That is, step 902 may also be performed before step 901.
Fig. 10 shows a schematic diagram of another method 1000.
The method 1000 is performed by a navigation device for providing navigation services to devices within an area. The method 1000 comprises the steps of:
step 1001: acquiring the current pose of the equipment;
step 1002: acquiring a map of a current subarea and at least one adjacent subarea according to the current pose;
Step 1003: acquiring one or more current transformations from a support device for supporting navigation services according to the current pose;
step 1004: transforming the map according to one or more current transforms;
step 1005: navigation services are provided according to the transformed map.
It should be noted that from the perspective of fig. 1-8 described above, the steps of method 900 and method 1000 may have the same functionality and detailed information. Accordingly, corresponding method implementations are not described in detail at this point.
An application scenario of the present invention may be to provide location services to a fleet of autonomous vehicles within an area. Each vehicle is equipped with one or more sensors, such as cameras, radar and GNSS modules. Map data, images, and coarse GNSS coordinates are collected in a crowdsourcing manner and sent to a remote server (which may be a support device). Several computing units (or staff members) may access the server in order to retrieve and process the map construction data.
The remote server divides the area into a plurality of appropriately sized sub-areas based on the complexity of the available computing resources or sub-areas. The map construction manager sends map construction instructions to a plurality of staff members. Each worker retrieves mapping data from remote servers located within the sub-area and runs a mapping algorithm instance to create a 2D/3D map of its own sub-area. The mapping manager and staff members may be integrated into the remote server or may be separate units from the remote server. For embodiments of the present invention, it is sufficient for the remote server to obtain a 2D/3D map of the divided sub-areas.
The remote server then collects the generated sub-maps from the staff member and initiates map stitching and transformation calculations. The global constraints applied to each sub-map are the GNSS coordinates and camera pose registered in the reconstructed 2D/3D model. GNSS coordinates attached to the corresponding camera pose result in a rigid transformation that aligns and scales the reconstructed 2D/3D model to the global frame of the corresponding GNSS position. The relative constraint between adjacent sub-maps is obtained by registering a plurality of 2D/3D images present in one sub-map to another adjacent sub-map. After registering the same image or the same features of both images for two adjacent sub-maps, the camera pose associated with the images may be transformed to the same pose, resulting in stronger relative constraints. Geofence transformations are calculated by considering global constraints and/or relative constraints and stored in a transformation database of a remote server.
To provide location services to a fleet of autonomous vehicles, the sub-maps and transformation databases stored on the remote server may be uploaded to the navigation unit of each vehicle. An extended kalman filter (Extended Kalman Filter, EKF) that fuses GNSS positions and poses obtained by PnP algorithm is used to estimate vehicle pose. The PnP algorithm uses matches between features extracted from images from one of the embedded cameras and the partially stitched sub-map. Using the EKF predicted filtered pose, map stitching is updated by determining the most recent reference position, retrieving the associated one or more transforms from the transform database.
Alternatively, the sub-map may be updated due to the continuous crowdsourcing data stream provided by the fleet of vehicles. If the geometry of the map is modified after the update, the geofence transformations around the sub-map are recalculated and the updated sub-map and transformation database are uploaded to the fleet.
One advantage of embodiments of the present invention is that positioning may be data independent. That is, the positioning of a splice-based neighboring map may be applied to LiDAR-based positioning or pure camera-based positioning (which may be categorized as map-based positioning).
Further, embodiments of the present invention are "method independent". That is, the sub-map may be constructed using different mapping algorithms, and different positioning algorithms may be used in navigating depending on the nature of the sub-map and the available sensors.
Further, embodiments of the present invention do not rely on a costly global alignment method, which typically does not converge to an optimal solution in the presence of small errors or drift in the individual sub-maps.
Further, the local map stitching and transformation computation in the embodiments of the present invention is robust to local map errors and map construction drift. Since sub-map alignment is calculated for many locations within the region of interest, drift that may have accumulated during the map construction process can be compensated for as the navigation device moves.
Further, embodiments of the present invention may create a distributed map faster and more efficiently. For example, SFM algorithms commonly used in the art to create maps are not well suited for use with a large number of images or points in a map. This is because the beam adjustment method (bundle adjustment, BA) step needs to be performed. BA is an operation with complexity of O ((nc+np) 3), where nc is the number of cameras in the model and np is the number of points in the model. Since the complexity is multiplied, the reduction of the model size due to the region division may have a positive effect.
Further, another advantage of embodiments of the present invention is that updating and maintaining a reduced-size map of a sub-region is much easier than managing a single large scale map of the entire region of interest. Local changes can only be applied to the relevant sub-map without affecting the entire positioning system.
Further, flexibility may be brought about in that different sources of map construction data may be used to create a map of a sub-region.
Further, computing resources may be saved by reusing the sub-map. For example, if two different regions of interest share one common region, a common sub-map between the regions of interest may be shared for the respective stitching processes.
It should be noted that the means of the present invention, including the support means and the navigation means, may comprise processing circuitry for performing, implementing or initiating, respectively, the various operations of the device described herein. The processing circuitry may include hardware and software. The hardware may include analog circuits or digital circuits, or both analog and digital circuits. The digital circuitry may include components such as application-specific integrated circuits (ASIC), field-programmable arrays (FPGA), digital signal processors (digital signal processor, DSP), or multi-purpose processors. In one embodiment, the processing circuitry includes one or more processors and a non-transitory memory coupled to the one or more processors. The non-transitory memory may carry executable program code that, when executed by one or more processors, causes the device to perform, implement, or initiate, respectively, the operations or methods described herein.
It should also be noted that the apparatus of the present invention may be a single electronic device capable of performing the calculations, or may comprise a connected set of electronic devices or modules capable of performing the calculations using a shared system memory. As is well known in the art, such computing capabilities may be incorporated into many different devices, so the term "device" may include chips, chipsets, computers (including on-board computers), servers, navigation devices, radar Microcontrollers (MCUs), advanced driving assistance systems (advanced driver assistance system, ADAS), autonomous vehicles, drones, mobile terminals, tablets, wearable devices, gaming machines, graphics processing units, graphics cards, and the like.
The present application has been described in connection with various embodiments and implementations as examples. However, other variations can be understood and effected by those skilled in the art by practicing the claimed subject matter, from a study of the drawings, the invention, and the independent claims. In the claims and in the description, the word "comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Claims (22)
1. An apparatus (310) for supporting intra-zone navigation, the apparatus being configured to:
generating one or more reference locations within the region;
dividing the region into a plurality of sub-regions (101, 103);
wherein for each reference position the device (310) is further configured to:
acquiring a map of a sub-region (101) in which the reference position is located and at least one adjacent map of at least one adjacent sub-region (103);
determining one or more transformations;
transforming one or more of the map and the at least one neighboring map according to the one or more transforms such that the map and the at least one neighboring map are aligned;
Associating the one or more transforms with the reference location;
the one or more transforms are provided to a navigation device (320).
2. The apparatus (310) according to claim 1, wherein the sub-region (101) and the at least one adjacent sub-region (103) share an overlapping portion (102).
3. The apparatus (310) according to claim 1 or 2, further configured to: the shape and/or size of each sub-region (101, 103) is determined according to one or more of the following criteria:
-the topography of the area;
-a geographical density of the environmental data of the area;
-available computing resources of the apparatus (310) for processing the environmental data associated with each sub-area (101, 103).
4. A device (310) according to any one of claims 1 to 3, characterized in that, in order to obtain the map of each sub-area (101, 103), the device (310) is adapted to:
obtaining raw data, the raw data comprising at least one of:
-one or more images;
-light detection and ranging LiDAR data;
-coordinate information of one or more global navigation satellite systems;
-geometric data;
-road data;
a representation of each sub-area (101, 103) is generated as the map from the acquired raw data.
5. The apparatus (310) of any of claims 1 to 4, wherein the map of each sub-area (101, 103) comprises at least one or more representations of:
-a two-dimensional 2D model of the sub-region (101, 103);
-a three-dimensional 3D model of the sub-region (101, 103);
wherein the 2D model is a 2D point cloud representing the sub-region (101, 103) or a 2D grid representing the sub-region (101, 103), and the 3D model is a 3D point cloud representing the sub-region (101, 103) or a 3D grid representing the sub-region (101, 103).
6. The apparatus (310) of claim 5, wherein the map of each sub-region (101, 103) further comprises pose information associated with the one or more representations.
7. The apparatus (310) according to claim 6 when dependent on claim 2, wherein to determine the one or more transformations, the apparatus (310) is specifically configured to:
a relative pose between the map and the at least one neighboring map is determined according to the overlapping portion (102), wherein the determined relative pose is one of the one or more transformations.
8. The apparatus (310) according to any one of claims 1 to 7, wherein to generate the one or more reference positions, the apparatus (310) is configured to: the one or more reference locations are generated at one or more landmarks within the region.
9. The apparatus (310) according to any one of claims 1 to 8, wherein to generate the one or more reference positions, the apparatus (310) is configured to: the one or more reference locations are generated within one or more accessible regions of the region.
10. The apparatus (310) according to any one of claims 1 to 7, wherein to generate the one or more reference positions, the apparatus (310) is configured to:
generating the one or more reference locations uniformly within the region; or (b)
The one or more reference locations are randomly generated within the region.
11. The apparatus (310) according to any one of claims 1 to 10, wherein the apparatus (310) is configured to: the number and/or distribution of the one or more reference locations is determined according to the complexity of the map of each sub-area (101, 103).
12. The apparatus (310) according to any one of claims 1 to 11, wherein to determine the one or more transformations, the apparatus (310) is configured to:
calculating local alignment transformation centering on the reference position between the map and the at least one adjacent map by adopting a stitching algorithm;
An output of the stitching algorithm is obtained as one of the one or more transforms.
13. The apparatus (310) of any of claims 1 to 12, wherein each transformation is a 2-degree-of-freedom, doF, or a 3DoF, or a 6DoF, or a 7DoF transformation, wherein the 2DoF transformation involves a two-axis translational transformation, the 3DoF transformation involves three-axis translational transformations, or the 2DoF transformation and direction, the 6DoF transformation involves three-axis rotational and translational transformations, and the 7DoF transformation involves three-axis rotational and translational transformations and scaling.
14. The apparatus (310) according to any one of claims 1 to 13, wherein to associate the one or more transformations with the reference position, the apparatus (310) is configured to: the one or more transforms are stored in a database (314) indexed according to the reference location.
15. The apparatus (310) according to any one of claims 1 to 14, wherein to provide the one or more transformations to the navigation device (320), the apparatus (310) is configured to:
receiving pose information from the navigation device (320);
determining a reference position according to the pose information;
The one or more transforms are determined based on the determined reference locations.
16. A navigation device (320) for providing navigation services to equipment within an area, characterized in that the navigation device (320) is adapted to:
acquiring the current pose of the equipment;
acquiring a map of a current subarea (101) and at least one adjacent subarea (103) according to the current pose;
acquiring one or more current transformations from a support device for supporting the navigation service according to the current pose;
transforming the map according to the one or more current transforms;
and providing the navigation service according to the transformed map.
17. The navigation device (320) of claim 16, further configured to:
updating the current pose;
updating the map and the one or more transforms according to the updated pose;
transforming the updated map according to the one or more updated transforms;
and continuing to provide the navigation service according to the updated and transformed map.
18. A system for in-area navigation, characterized by comprising a support device (310) for supporting navigation according to any of claims 1 to 15 and a navigation device (320) according to claim 16 or 17.
19. A method (900) for supporting intra-area navigation, the method being performed by a supporting device (310) and comprising:
generating (901) one or more reference locations within the region;
dividing (902) the region into a plurality of sub-regions;
for each reference position, the method further comprises:
acquiring (903) a map of a sub-region in which the reference position is located and at least one adjacent map of at least one adjacent sub-region;
determining (904) one or more transformations;
transforming (905) one or more of the map and the at least one neighboring map in accordance with the one or more transforms such that the map and the at least one neighboring map are aligned;
associating (906) the one or more transforms with the reference location;
the one or more transforms are provided (907) to the navigation device (320).
20. A method (1000) for providing navigation services to devices within an area, characterized in that the method is performed by a navigation device (320) and comprises:
acquiring (1001) a current pose of the device;
acquiring (1002) a map of a current sub-region (101) and at least one adjacent sub-region (103) according to the current pose;
Acquiring (1003) one or more current transformations from a support means for supporting the navigation service, in accordance with the current pose;
transforming (1004) the map according to the one or more current transforms;
-providing (1005) the navigation service according to the transformed map.
21. A computer program product, comprising: program code for performing the method according to claim 19 when executed on a computer.
22. A computer program product, comprising: program code for performing the method according to claim 20 when executed on a computer. .
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/EP2021/071362 WO2023006213A1 (en) | 2021-07-30 | 2021-07-30 | Device and method for navigation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117730239A true CN117730239A (en) | 2024-03-19 |
Family
ID=77338648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202180100994.2A Pending CN117730239A (en) | 2021-07-30 | 2021-07-30 | Apparatus and method for navigation |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4281729A1 (en) |
| CN (1) | CN117730239A (en) |
| WO (1) | WO2023006213A1 (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9140792B2 (en) * | 2011-06-01 | 2015-09-22 | GM Global Technology Operations LLC | System and method for sensor based environmental model construction |
| US10837773B2 (en) * | 2016-12-30 | 2020-11-17 | DeepMap Inc. | Detection of vertical structures based on LiDAR scanner data for high-definition maps for autonomous vehicles |
| US10317220B2 (en) * | 2017-08-30 | 2019-06-11 | Aptiv Technologies Limited | Alignment of multiple digital maps used in an automated vehicle |
| US10529089B2 (en) * | 2018-02-23 | 2020-01-07 | GM Global Technology Operations LLC | Crowd-sensed point cloud map |
| CN110849380B (en) * | 2019-10-28 | 2022-04-22 | 北京影谱科技股份有限公司 | Map alignment method and system based on collaborative VSLAM |
| WO2021090971A1 (en) * | 2019-11-06 | 2021-05-14 | 엘지전자 주식회사 | Path providing device and path providing method thereof |
| CN111656138A (en) * | 2020-04-16 | 2020-09-11 | 珊口（上海）智能科技有限公司 | Map construction and positioning method, client, mobile robot and storage medium |
-
2021
- 2021-07-30 WO PCT/EP2021/071362 patent/WO2023006213A1/en active Application Filing
- 2021-07-30 CN CN202180100994.2A patent/CN117730239A/en active Pending
- 2021-07-30 EP EP21754949.2A patent/EP4281729A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| EP4281729A1 (en) | 2023-11-29 |
| WO2023006213A1 (en) | 2023-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10659925B2 (en) | Positioning method, terminal and server | |
| KR100728377B1 (en) | Method for real-time updating gis of changed region vis laser scanning and mobile internet | |
| US11549817B2 (en) | Distributed device mapping | |
| JP2019179021A (en) | Method and apparatus for creating map and positioning moving entity | |
| Cappelle et al. | Virtual 3D city model for navigation in urban areas | |
| WO2019099802A1 (en) | Iterative closest point process based on lidar with integrated motion estimation for high definitions maps | |
| WO2018125939A1 (en) | Visual odometry and pairwise alignment for high definition map creation | |
| CN111383205B (en) | Image fusion positioning method based on feature points and three-dimensional model | |
| KR102200299B1 (en) | A system implementing management solution of road facility based on 3D-VR multi-sensor system and a method thereof | |
| KR101444685B1 (en) | Method and Apparatus for Determining Position and Attitude of Vehicle by Image based Multi-sensor Data | |
| JP2022077976A (en) | Image-based positioning method and system | |
| CN112414417B (en) | Automatic driving map generation method and device, electronic equipment and readable storage medium | |
| WO2021254019A1 (en) | Method, device and system for cooperatively constructing point cloud map | |
| CN116295457B (en) | Vehicle vision positioning method and system based on two-dimensional semantic map | |
| US20230334850A1 (en) | Map data co-registration and localization system and method | |
| JP2019537018A (en) | Method and system for generating an environment model and for positioning using a mutual sensor feature point reference | |
| Ishikawa et al. | A study of precise road feature localization using mobile mapping system | |
| CN117730239A (en) | Apparatus and method for navigation | |
| US11747141B2 (en) | System and method for providing improved geocoded reference data to a 3D map representation | |
| CN117635721A (en) | Target positioning method, related system and storage medium | |
| CN114842224A (en) | Monocular unmanned aerial vehicle absolute vision matching positioning scheme based on geographical base map | |
| Kurdi et al. | Navigation of mobile robot with cooperation of quadcopter | |
| CN113960614A (en) | Elevation map construction method based on frame-map matching | |
| CN112400122A (en) | System and method for locating target object | |
| CN117330097B (en) | Vehicle positioning optimization method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |