DE102022201427A1 - Method for determining a position and/or orientation of a mobile device in an environment - Google Patents

Abstract

The invention relates to a method for determining a position and/or orientation of a mobile device (100) in an environment (150) in which the mobile device (100) moves or is intended to move, comprising: providing a current first set of points first nature in the environment determined from and relative to the mobile device (100); providing a current second set of one or more tracked markers (162, 164, 165); determining a third set of tracked markers (162, 164, 165) included in both the current second set and a second reference set of one or more tracked markers (160-165); and determining a current position and/or orientation of the mobile device (100) in the environment (150) based on the current first set, a first reference set of points of the first type, the third set and the second reference set. The invention also relates to a mobile device (100).

Description

The present invention relates to a method for determining a position and/or orientation of a mobile device, e.g. a partially automated moving vehicle or a robot, in an environment, a computing unit and a computer program for its implementation, and such a mobile device.

Background of the Invention

Mobile devices such as vehicles or robots that move at least partially automatically move along a movement path in an environment such as an apartment, in a garden, in a factory building or on the street, in the air or in water. For this it is necessary that a current position and/or orientation (also referred to as a pose) of the mobile device is known and is also determined again and again. For this purpose, e.g. a laser distance measurement (e.g. with so-called lidar sensors) can be used.

Disclosure of Invention

According to the invention, a method for determining a position and/or orientation of a mobile device, a computing unit and a computer program for executing it, and such a mobile device with the features of the independent patent claims are proposed. Advantageous configurations are the subject of the dependent claims and the following description.

The invention deals with determining a (current) position and/or orientation (also referred to as a pose) of a mobile device in an environment, ie in particular the localization of the mobile device. A possible mobile device here is, for example, a vehicle that moves at least partially automatically, in particular a passenger transport vehicle or goods transport vehicle (e.g. forklift truck, industrial truck), or a robot. The principle of distance measurement using a laser (laser distance measurement, e.g. with so-called lidar sensors) is used. For this purpose, one or possibly several such lidar sensors are typically provided on or in the mobile device, with which distances (distances) of objects in the environment (relative) to the mobile device (or to the sensor) can be determined. Such a lidar sensor typically scans the area in front of the mobile device at a certain angle, e.g. 90°, 180°, 270° or even more.

A more historical option for laser distance measurement and localization based on it (laser localization), which is nowadays increasingly to be regarded as outdated, is based on the exclusive use of so-called reflector markers, which have to be installed across the board in the surrounding area. These reflector markers can be recognized by the laser scanner or lidar sensor in that points measured on them are reflected with a significantly higher intensity than in the rest of the environment. Laser scanners can, for example, recognize the reflector markers themselves if desired, i.e. only measured values that correspond to a certain intensity (e.g. higher than a certain threshold value) can be taken into account directly.

Once the reflector markers have been attached, their position can be measured manually and entered on a map. During localization, the positions of recognized reflector markers can be constantly compared with the (then precisely known) positions of the reflector markers on the map. If at least two of the mapped reflector markers are always recognized, the pose (position and orientation) of the laser scanner can be clearly determined. In particular, it should be assumed that the mobile device and the laser scanner move in a 2D plane, i.e. only have three degrees of freedom (two translational and one rotational degree of freedom). In principle, however, the principle can also be used in 3D, for example.

However, attaching and measuring reflector markers is usually very time-consuming. The computing power of computing units or computing systems, even in embedded computer systems such as those used in mobile devices, has increased significantly in recent times. As a result, the localization (determination of position and/or orientation) of mobile devices using natural contours with the help of so-called "scan matching" has become established in recent years. As a result, reflector markers have been superseded.

Specifically, “scan matching” uses the entire point cloud measured or captured by the laser scanner; A point cloud is a set of points in the environment that are determined using the laser scanner or lidar sensor. A distance from the mobile device or laser scanner can be assigned to each point, as can an orientation relative to a reference orientation of the mobile device or laser scanner. The point cloud usually corresponds to one or more "point lines" along the contour of the objects in the field of view; if necessary, however, the points can only lie approximately on such a line.

However, this point cloud alone does not yet allow a position and/or orientation of the mobile device to be determined in the environment. For this purpose, the point cloud or set of points is compared with a reference point cloud or reference set of points. A transformation can then be determined which best brings the point cloud (or set of points) into congruence with the reference point cloud (or reference set of points), i.e. aligns both with one another. This transformation then corresponds to a position and/or orientation of the mobile device when the point cloud is recorded relative to the reference point cloud or a coordinate system of the reference point cloud. If the reference point cloud is a map of the area or at least a part of it, then the current position and/or orientation of the mobile device in the area can be determined. The reference point cloud (or map of the environment) can be expanded, for example, by constantly or repeatedly adding new point clouds or parts thereof.

SLAM is also used in this context. SLAM (“Simultaneous Localization and Mapping”) is a method in robotics in which a mobile device like a robot can simultaneously create a map of its surroundings and estimate its spatial position within this map or must. It is used, for example, to recognize obstacles and thus supports autonomous navigation.

In order to be able to determine the position and/or orientation of the mobile device in the environment with sufficient accuracy in this way, the environment should have a sufficient number of characteristic contours that are reflected in corresponding points in the point cloud to solve the problem underlying "scan matching". in all three degrees of freedom. However, as has been shown, there are cases where this is not the case.

Against this background, it is proposed to also consider reflector markers or other markers in addition to the set of points as in the usual "scan matching".

In this case, a current first set of points of the first type in the area is provided or obtained, which have been determined on the basis of and relative to the mobile device. A distance measurement (e.g. the already mentioned laser scanning) can be used for this purpose, e.g. by means of a lidar sensor provided on or in the mobile device. These points of the first type are, in particular, points as in the usual “scan matching” as mentioned above.

A current second set of one or more markers is also provided or obtained. In particular, the current second set contains the currently “visible” markers. Furthermore, a third set of tracked markers is determined, which is contained both in the current second set and in a second reference set of one or more tracked markers. This can be done, for example, using identifications (or IDs) that are or will be assigned to the individual (tracked) markers, as will be explained in more detail later. Tracked markers are to be understood in particular as markers in the environment, e.g. the reflector markers mentioned, which have already been recognized in earlier scans than the current scan (or acquisition cycle), i.e. e.g. tracked over several scans. More details on this will follow later. Markers are preferably determined using second-type points, such that second-type points correspond to the markers, with first-type points corresponding to objects in the environment other than the markers. A distance measurement (e.g. the already mentioned laser scanning) can also be used for points of the second type, in particular by means of the same laser scanner or lidar sensor as for the points of the first type.

The points of the second type are then in particular those which correspond to a light intensity determined during the distance measurement which is above a threshold value. The threshold value can be fixed, for example. However, a distinction can also be made in other ways, for example with a threshold value that is dependent on the median or mean value of all intensities of a scan, or by means of machine learning algorithms. However, it is also particularly useful that an intensity jump at the edge of the marker is taken into account, on the same spatial level, from low (where there is no marker) to high (where the marker begins) and back to low (where the marker ends again). The points of the first type can be all points or only those that correspond to a light intensity determined during the distance measurement that is below the threshold value.

The points of the second type can therefore correspond to predefined markers such as the aforementioned reflector markers in the area, while the points of the first type can correspond to all or all other objects in the area than the predefined markers. It should be noted that - e.g. depending on the size or degree of coverage of a marker - one or more points of the second type can correspond to a single marker. There may be points of the second type corresponding to one or more markers in the relevant set of a scan. If there are several markers, there are then necessarily several points of the second type.

A current position and/or orientation of the mobile device in the environment is then determined based on the current first set, a first reference set of points of the first type, the third set and the second reference set. This includes, in particular, that the first quantity is brought into agreement with the first reference quantity as part of an optimization, and that a cost function, in particular weighted, is taken into account in the optimization. The cost function gives a measure of distances of the tracked markers between the third set and the second reference set. It goes without saying that--as is usual for an optimization--an exact match is not or does not have to be achieved as a rule; rather, an attempt is made to achieve as good a match as possible. Based on the current position and/or orientation of the mobile device, a movement path can then also be determined along which the mobile device is to move in the area. For this purpose, control information for moving the mobile device can be determined based on the movement path and the current position and/or orientation; the control information can then be provided or the mobile device can be moved based on the control information.

The current position and/or orientation of the mobile device in the area can be used, especially in the case of non-automated mobile devices, e.g. to track the mobile device. For example, it can be determined where a forklift or other mobile device is currently located within a factory building.

The "scan matching" (with natural contours of objects in the environment, points of the first kind) is thus combined here with the idea of reflector markers in laser distance measurement (or generally markers or points of the second kind) in order to combine the advantages of both approaches and balance the disadvantages of both approaches. For example, localization using "scan matching" can generally be relied on, at least where this works with sufficient reliability; this is typically the case in more than 95% of environments. However, if problematic areas are to be found in the desired environment, a user can, for example, attach a certain number of reflective markers, the use of which then makes the possibly previously ambiguous result of the “scan matching” clear at this point. Such problematic areas can be, for example, corridors with little structure, areas with uneven ground or environments with a large number of dynamic objects.

The advantage compared to localization without reflector markers is a significantly improved reliability of the localization result in problematic areas. The advantage compared to classic purely reflector marker-based localization is the significantly lower effort when attaching markers. Instead of a large number of markers all over the area, only a few markers have to be attached at critical points, for example.

Preferably, the first set is matched to the first reference set by determining a transformation that maps the first set to the first reference set, wherein the current position and/or orientation of the mobile device in the environment is determined based on the transformation. The transformation is preferably also taken into account in the cost function. In practice, often no transformation will be found that causes an exact mapping of the sets to each other; Rather, it is about the best possible transformation - within the framework of a time-limited optimization. In particular, when determining the transformation, points of the second type or markers are weighted more highly than points of the first type, especially since these are typically significantly more accurate. Thus, two sets of dissimilar points can be treated simultaneously during a "scan matching" process within a laser localization system, i.e. all conventional points (points first type) on natural contours on the one hand, and points on recognized reflector markers (points of the second type) on the other hand.

While even the best "scan matching" result using only contour points (points of the first kind) can in many cases be ambiguous - for example the position in a long corridor from which only two parallel lines can be seen, thus not being recognized with certainty where exactly the position is along the corridor - the inclusion of points on reflector markers or points recognized on them (points of the second type) can lead to a clear result. The "scan matching" problem is then solved taking into account the markers or the points recognized on them.

As a "scan matching" method, i.e. a method in which the first set is mapped to the first reference set, or in which a transformation is determined that causes this mapping, fundamentally different types of methods or processes can be considered. For example, this can be the so-called “Iterative Closest Point” method (ICP) or the “Normal Distributions Transform” method (NDT), see P. Biber and W. Strasser, “The normal distributions transform: a new approach to laser scan matching,” Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), 2003, pp. 2743-2748 vol.3, doi: 10.1 109/IROS.2003.1249285. The tracked markers or the points of the second type can be taken into account in a particularly simple manner as a cost function (or cost term) as part of the "scan matching" method for the first set or reference set.

Within the scope of the present invention, special emphasis is placed on the tracking of markers. Markers of the current second set (i.e. in the current scan), which are also present in the second reference set, i.e. had been recognized earlier, are used to determine the position and/or orientation and are referred to here as the third set.

Preferably, markers of the current second set that are not contained in the third set, ie have not yet been recognized, are added after the position and/or orientation of the second reference set has been determined. In future scans, these will also serve as previously recognized - tracked - markers.

It is preferably also recorded (and stored) how often a marker from the second reference set has already been recognized. This can be used, for example, to increase the accuracy if only those markers from the second set are assigned to the third set, for which this number has already reached at least a detection threshold value, e.g. three times.

In other words, the following ("tracking") of markers or reflector markers is used in order to be able to use the information from it later for "scan matching". From the individual detections of the markers per scan, connected tracks (hits), i.e. markers tracked (over several scans), can be generated. It is taken into account here that the change in the position and/or orientation of the mobile device usually deviates only slightly in the case of successive scans, at least less than in the case of scans that are more distant in time. This allows markers to be tracked; such markers are then very definitely also markers actually present in the environment and can have a positive influence on the localization. An even more detailed explanation of the tracking of markers should be given in relation to the description of the figures.

At least some of the points of the current first set are preferably added to the first reference set if at least one criterion is met. The first criterion can e.g. require that at least a certain proportion of the points of the current first set, e.g. 25%, 50% or even more, is not already present in the first reference set. In this way, the first reference set—and ultimately also the actual map of the environment—can be expanded. With each new, up-to-date first set, the map of the area can then be further improved, if necessary. Other possible criteria (heuristics) are, for example, whether a certain distance has been covered or whether the NDT score is below a certain threshold.

A further preferred criterion for this results from the tracking of the markers. As soon as a new tracked marker is created, ie added to the second reference set, at least part of the points of the current first set or all of the current first set can be added to the first reference set. The part contains in particular the newly tracked marker or the area around it, the entire first set contains this anyway. This allows future "Scan Matches" use this marker, since it then occurs both in the current scan and in the local map (reference set).

Since the local map is typically composed of several scans, it can happen that a tracked marker appears multiple times in the local map, i.e. that there are several tracked markers with the same ID. There are several ways to deal with this: The oldest tracked marker can be used to reduce drift. The latest tracked marker can be used to be more robust against markers that are not completely static. An average of all tracked markers with the same ID can be used to become more robust to noisy marker position measurements.

It is also advantageous if the current position and/or orientation of the mobile device in the area is only determined based on the current first set, the first reference set, the third set and the second reference set if another criterion is met . This additional criterion can, for example, require that a marker (or point of the second type) is present in the current measurement. Alternatively or additionally, the further criterion can require, for example, that the mobile device is located within a certain area of the environment, e.g. one in which it is known that reflector markers are present. If the other criterion is not met, the current position and/or orientation of the mobile device in the environment is determined based only on the first set and the first reference set, i.e. as with conventional "scan matching" (without reflector markers or similar ).

A computing unit according to the invention, e.g. a control unit of a robot, is set up, in particular in terms of programming, to carry out a method according to the invention.

The invention also relates to a mobile device with a computing unit according to the invention. The mobile device is preferably designed as a vehicle that moves at least partially automatically, in particular as a passenger transport vehicle or as a goods transport vehicle, or as a robot.

The implementation of a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is advantageous because this causes particularly low costs, especially if an executing control device is also used for other tasks and is therefore available anyway. Suitable data carriers for providing the computer program are, in particular, magnetic, optical and electrical memories, such as hard drives, flash memories, EEPROMs, DVDs, etc. It is also possible to download a program via computer networks (Internet, intranet, etc.).

Further advantages and refinements of the invention result from the description and the attached drawing.

It goes without saying that the features mentioned above and those still to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

The invention is shown schematically in the drawing using an exemplary embodiment and is described in detail below with reference to the drawing.

character list

• 1 shows schematically a mobile device in an environment in which a method according to the invention can be carried out.
• 2 shows schematically reference sets of points as can occur in a preferred embodiment of a method according to the invention.
• 3 shows schematically sets of points as can occur in a preferred embodiment of a method according to the invention.
• 4 shows schematically a sequence of a method according to the invention in a preferred embodiment.

Detailed description of the drawing

In 1 a mobile device 100 is shown schematically in an environment 150 in which a method according to the invention can be carried out. The mobile device 100 is, for example, a robot or an automated (or autonomous) driving vehicle, with a computing unit 110 and a lidar sensor 120 for laser distance measurement; a detection range of the lidar sensor 120 is indicated by dashed lines as an example.

The environment 150 is, for example, the interior of a building with a longer corridor, for example, from the left in the 1 beginning with side walls 152 and 154. The hallway widens, to the right in the 1 towards, towards the side wall 152 side, two columns 156 and 158 are provided. Three reflector markers 160, 161, 162 or 163, 164, 165 are provided on the side walls 152 and 154, for example.

The mobile device 100 is now supposed to move along the movement path 102 in the environment 150, for example. This requires knowledge of the current position and/or orientation of the mobile device 100 in order to be able to determine how the mobile device 100 must be controlled in order to be able to follow the movement path 102, or also in order to be able to plan the movement path 102 first . As mentioned, knowledge of the current position and/or orientation can also be used (only) to track the mobile device.

For this purpose, the environment 150 can be scanned by means of the lidar sensor 120 . The laser scanner is moved in the lidar sensor, while a large number of points are obtained by means of emitted laser beams and received, reflected radiation; each point stands for a point or a small area in the environment, e.g. on an object from which reflected radiation is received.

In 2 are off for this for the environment 150 1 points are shown as examples of how they can be obtained in principle using laser distance measurement. There are points 210 of the first type, which arise or are obtained by reflection of the laser beams on, for example, the side walls 152, 154 and the side surfaces of the columns 156, 158. These points 210 of the first type are represented by small crosses. There are also points 220 of the second type, which arise or are obtained by reflecting the laser beams at the reflector markers 160 to 165 . These points 220 of the second type are represented by circles. As mentioned, several points of the second type can also be present for each reflector marker.

These two types of points can be distinguished because the radiation reflected by the reflector markers generates a significantly higher light intensity than the radiation reflected by the other objects in the area, due to the corresponding design of the reflector markers (e.g. as mirrors) in the lidar sensor, where Laser radiation is mostly diffusely scattered.

In the 2 The points shown at least essentially form the surroundings 150 1 away; it is more points than seen from the mobile device 100 at the in 1 shown position can be detected. These points can serve as a reference or map of the area 150; it may have been obtained over several acquisition cycles, for example. For this purpose, the points 210 of the first type can be combined to form a first reference set 212 .

In 3 are off for the area 150 1 points are also shown as examples of how they can in principle be obtained by means of laser distance measurement. There are points 310 of the first type, which arise or are obtained by reflection of the laser beams on, for example, the side walls 152, 154 and the side surfaces of the columns 156, 158. These points 310 of the first type are represented by small crosses. There are also points 320 of the second type, which arise or are obtained by reflecting the laser beams at the reflector markers 162, 164 and 165. These points 320 of the second type are represented by circles.

Unlike in 2 , those in 3 points shown only by those points which, from the point of view of the mobile device 100, are at the in 1 shown position and can be detected in about the orientation shown there. These points can be regarded as current points, such as can be acquired or obtained at a specific position with a specific orientation of the device. The points 310 of the first type can be combined into a first set 312 .

In order to be able to use the information from the markers or reflector markers later for “scan matching”, the tracking of markers or reflector markers is used. From the individual detections of the markers per scan, connected tracks (hits), i.e. markers tracked (over several scans), can be generated.

It can be assumed that for each scan (i.e. in each time step) there is a set of detections D = {d ₁ , ... , d _{N D} } are given as the second set and a set of markers (“tracks”) T = {t ₁ , ..., t _NT } that have been tracked (ie recognized earlier or more than once) as the second set of references. The set of tracked markers includes, for example, those markers that were already present in one or more previous scans. For example, the six markers 160 to 165 can do this 1 be such that the six points 220 of the second type shown serve as a second reference set 222 of tracked markers ( 2 ) can be viewed.

The set of detections includes those markers that are detected in the current scan. For example, the three markers 162, 164 and 165 can do this 1 be such that the three points 320 shown of the second type as a second set 322 of markers ( 3 ) can be viewed.

In order to determine those markers that are used for pose determination, detection and tracks are associated with each other such that the sum of the distances between the tracks and the detections, cost(A), is minimized. So, for example, you can try to use the three markers that are in 3 have been recognized, the six in 2 assigned to recognized markers.

Each detection and each track can be assigned a maximum of once. For example, detections and tracks that remain have fixed costs c _D > 0 or c _T > 0. The set of assignments is described as A = {(i,j) | d _i is associated with t _j }. $$\text{cost}\left(\text{A}\right)={\displaystyle \sum _{\left(i,j\right)\in A}{\left|{\mathrm{i.e}}_i-t_j\right|}^2+\left(N_D-\left|A\right|\right)\cdot c_D+\left(N_T-\left|A\right|\right)\cdot c_T}$$

N _D and N _T are the number of unassigned detections or tracks. The allocations that minimize these costs are e.g. B. with the "Hungarian Method" (Hungarian Algorithm, also Kuhn-Munkres Algorithm, cf. "James Munkres: Algorithms for the Assignment and Transportation Problems, Journal of the Society for Industrial and Applied Mathematics, Vol. 5 No. 1, 32-38, 1957.”). A new track is then created for unassigned detections and provided with a unique track ID, ie they are added to the second reference set. For example, if the marker is 165 in 2 had not (yet) been detected, a new track or a new tracked marker would be created for this. A track can in particular include coordinates of the marker.

Unassigned tracks, e.g. markers 160, 161, 163, which are in 2 have still been recognized initially remain unchanged, but are discarded, for example, after a certain number of time steps or scans without an associated detection.

In order to increase the robustness against short-term incorrect detections (e.g. incorrect detections that only appear in a single scan), it is advisable to only use tracks for further processing that have been detected several times, e.g. at least three times.

To the position and orientation of the mobile device in the situation of 3 to determine, the first set 312 is to be made to overlap or match with the first reference set 212, with the markers or tracks that are contained in both the current second set 322 and in the second reference set 322 - these are defined as third set 332 and in the example shown the markers 162, 154, 165 are taken into account via a cost function. A transformation that, for example applied to the first set, maps the corresponding points to the first reference set, yields the position and orientation of the mobile device in the situation 3 relative to the situation in 2 .

In 4 a method according to the invention is now shown schematically in a preferred embodiment using a flow chart. In a new measurement cycle, information can be obtained or recorded, for example from odometry and/or inertial sensors, in a step 400, which allows the current position and/or orientation (at least roughly) to be estimated in a step 402. This is possible, for example, because odometry can be used to determine how much the posi tion has changed relative to the last known position; this can be determined for orientation using inertial sensors. If such information is not available or cannot or should not be used, the current position and/or orientation can also be estimated using empirical values, for example. This estimated current position and/or orientation can then be used for the subsequent "scan matching" procedure. However, the last known position and/or orientation can also be used for the subsequent “scan matching” method, ie such an initial estimation does not necessarily have to take place.

Furthermore, in a step 404 a scan is carried out using the lidar sensor. In this case, in step 406, the current first set 312 of points 310 of the first type is obtained. In step 408, one or more points 320 (in the example shown in 3 there are three points) of the second kind received.

Based on the one or more points 320, in a step 410 a current second set of tracked markers is determined. From this, in a step 412, a third set of tracked markers is determined, which are contained both in the current second set and in the second reference set,

Then, in step 414, a "scan matching" method is applied. In this case, the estimated or possibly last known position and/or orientation is used. As already mentioned, there are different so-called "scan matching" methods (or point cloud registration methods), most of which are based on either the "Iterative Closest Point" method (ICP) or the "Normal Distributions Transform" method (NDT) based. In the following, the NDT method for "Scan Matching" will be explained in more detail as an example.

First of all, the functionality of a "scan matching" method should be briefly explained. A current set of points or a point cloud P _A = {p _A1 ,...,>p _Ai }, reference set, is input to a "scan matching" procedure (this applies to all "scan matching" procedures). at points or a reference point cloud P _R = {p _R1 , ..., p _Rj } and an initial estimate of the position and/or orientation of the current point cloud T _R,A relative to the reference point cloud (or the last known one, if applicable). .

In a first step, the NDT "Scan Matching" process converts the reference point cloud into a so-called NDT representation. This is a discretized grid of cells (NDT cells), each of which is either empty or approximates the points they contain using a multivariate normal distribution with mean q and covariance matrix Σ. In order to avoid discretization effects, it is advisable to use several overlapping grids.

In a second step, all points of the current point cloud are transformed into reference coordinates based on the current or initial estimate of the position and/or orientation (pose estimate) T _R,A and with the next NDT cell or its mean q _i and covariance matrix Σ _i associated.

In a third step, the relative position and/or orientation (pose) T _Rr4 that maximizes the NDT score is determined: $$\text{NDT}-\text{score}={\displaystyle \sum _i\text{ex}}\left(-\frac{1}{2}{\left(T_{R,A}{p}_{A_i}-q_i\right)}^t{\displaystyle {\sum }_i^{-1}\left(T_{R,A}{p}_{A_i}-q_i\right)}\right)$$

The second and third steps are usually repeated several times, since changed pose estimates can lead to different associations between points and NDT cells. That relative position and/or orientation (pose) T _R,A that maximizes this NDT score then represents the current position and/or orientation.

This "scan matching" method can be carried out for the mentioned current first set of points of the first type and the first reference set of points of the first type, with the tracked markers having to be taken into account as a cost function, as will be explained below.

When using markers or reflector markers, in addition to the points (first type) P _A and P _R mentioned above, there is also the second set of markers T _AM = {t _{AM 1} , ...,t _{AM k} } and the corresponding second reference set T _RM = {t _{RN 1} ,...,t _{RN l} } available. The second set of markers T _AM can correspond to the above-mentioned set of detections D, but in particular can also only be a subset thereof. The (tracked) markers can have IDs or track IDs M ₁ . . . , M _k or N ₁ , . . . , N _l . The track IDs can be used to determine which markers appear in both sets; this is then the third set mentioned. These are then those markers that have been recognized in the current scan and also in previous scans. Based on this, a cost function can then be defined: $${\text{cost}}_{\text{marker}}={\displaystyle \sum _{k\in \left\{M_1,...,M_k\right\}\cap \left\{N_1,...N_l\right\}}{\left|T_{R,A}{t}_{A,k}-t_{R,k}\right|}^2}$$

wobei der Parameter w_marker > 0 die Gewichtung von Reflektormarkern im Verhältnis zum konturbasierten Matching (Punkte erster Art) angibt. Die Ableitung von cost_marker und somit auch der gesamten Scorefunktion, Gesamt-Score, lässt sich analytisch bestimmen, so dass die optimale Pose wie im gewöhnlichen „Scan Matching“ z.B. mit dem Newton-Verfahren bestimmt werden kann.Using this cost function, the NDT score can be adjusted to obtain an overall score: $$\text{In total}-\text{score}=\text{NDT}\_\text{score}-{\text{w}}_{\text{maker}}\cdot {\text{cost}}_{\text{marker}}$$

where the parameter w _marker > 0 specifies the weighting of reflector markers in relation to contour-based matching (points of the first kind). The derivation of cost _marker and thus also of the overall score function, overall score, can be determined analytically, so that the optimal pose can be determined as in the usual “scan matching”, for example with the Newton method.

In a step 416, the current position and/or orientation (pose) 418 can then be output. Based on this, a movement path can also be determined in a step 420, based on which in turn, in step 422, control information for moving the mobile device is determined. to move this.

In addition, in a step 424, at least a part of the points of the current first set can be added to the first reference set, e.g. if a certain distance has been covered or the NDT score is below a certain threshold or a new tracked marker is created.

Claims (15)

A method for determining a position and/or orientation of a mobile device (100) in an environment (150) in which the mobile device (100) moves or is intended to move, comprising: Providing (406) a current first set (312) of points (310) of the first type in the environment, which have been determined starting from and relative to the mobile device (100), providing (410) a current second set (322) of one or more markers (162, 164, 165), Determining (412) a third set (332) of tracked markers (162, 164, 165) that are in both the current second set (322) and in a second reference set (222) of one or more tracked markers (160-165 ) are included, and Determining (416) a current position and/or orientation (418) of the mobile device (100) in the area (150) based on the current first set (312), a first reference set (212) of points (210, 310) first type, the third set (332) and the second reference set (222). procedure after claim 1 , wherein the determination of the current position and / or orientation of the mobile device (100) comprises that the first set (312) with the first reference set (212) is brought into line as part of an optimization, and that in the optimization a Cost function, in particular weighted, is taken into account, the cost function indicating a measure of the distances between the tracked markers between the third set (332) and the second reference set (222). procedure after claim 2 , wherein the first set (312) is matched with the first reference set (212) by determining a transformation that maps the first set (312) to the first reference set (212), and wherein the current position and/or Orientation (414) of the mobile device (100) in the environment (150) is determined based on the transformation, and wherein the transformation is preferably taken into account in the cost function. Method according to one of the preceding claims, further comprising after determining (416) the current position and/or orientation (418) of the mobile device (100): adding markers of the current second set (322) not contained in the third set to the second reference set (222). A method according to any one of the preceding claims, wherein markers are or have been determined using second type points (320) such that second type points correspond to the markers, and wherein Points (210, 310) of the first type correspond to objects in the environment other than the predefined markers. Method according to one of the preceding claims, wherein points (210, 310) of the first type and/or points (220, 320) of the second type are or have been determined by means of distance measurement, in particular by means of a lidar sensor (120) of the mobile device (100). are. procedure after claim 6 , wherein points (220, 320) of the second type correspond to a light intensity determined during the distance measurement that is above a threshold value, and wherein points (210, 310) of the first type correspond to a light intensity determined during the distance measurement that is below a threshold value. Method according to one of the preceding claims, wherein at least part of the points of the current first set (310) is added to the first reference set (210) if at least one criterion is met, which preferably includes a criterion that requires that the second set a new tracked marker is added. A method according to any one of the preceding claims, further comprising: Determining (420) a movement path (102) on which the mobile device (100) should move in the environment (150) based on the current position and/or orientation (414) of the mobile device. procedure after claim 9 , further comprising: determining (422), based on the movement path (102) and the current position and/or orientation (414), control information for moving the mobile device (100), and providing the control information and/or moving the mobile device (100) based on the control information. Arithmetic unit (110) which is set up to carry out the method steps of a method according to one of the preceding claims. Mobile device (100) with a computing unit (110). claim 11 . Mobile device (100) after claim 12 , which is designed as an at least partially automated moving vehicle, in particular as a passenger transport vehicle or as a goods transport vehicle, or as a robot. Computer program that causes a computing unit (110) to use a method according to one of Claims 1 until 10 to be performed when it is executed on the computing unit (110). Machine-readable storage medium with a computer program stored on it Claim 14 .

Images