DE102020213133A1 - Method and device for evaluating a self-localization of a physical system using sensor data - Google Patents

Abstract

Device (100) for carrying out a localization using sensor data, comprising:- a first determination device (10) for determining a first pose (P1) from first sensor data;- a second determination device (20) for determining a second pose (P2) from second sensor data ; and- a verification device (30) for verifying the first pose (P1) determined by the first determination device (10), the verification device (30) having:- a first comparison device (31) for comparing the first pose (P1) with the second pose (P2), a confidence signal (K) relating to a confidence in the self-localization carried out being able to be output by means of the verification device (30) as a function of the comparison carried out.

Description

The invention relates to a device for evaluating a self-localization of a physical system using sensor data. The invention also relates to a method for evaluating a self-localization of a physical system using sensor data. The invention also relates to a computer program product.

State of the art

DE 10 2017 221 691 A1 discloses a method for self-localization of a vehicle in its environment.

DE 10 2015 211 279 A1 discloses a method for checking position signals of a global satellite navigation system for plausibility in a signal receiver, in particular vehicles or land vehicles.

EP 3 462 134 A1 discloses autonomous vehicles and, more particularly, a method and system for correcting a pre-generated navigation path for an autonomous vehicle.

Disclosure of Invention

It is an object of the present invention to provide an improved device for self-localization using sensor data.

• - eine erste Ermittlungseinrichtung zum Ermitteln einer ersten Pose aus ersten Sensordaten;
• - eine zweite Ermittlungseinrichtung zum Ermitteln einer zweiten Pose aus zweiten Sensordaten; und
• - eine Verifizierungseinrichtung zum Verifizieren der von der ersten Ermittlungseinrichtung ermittelten ersten Pose, die Verifizierungseinrichtung aufweisend:
• - eine erste Vergleichseinrichtung zum Vergleichen der ersten Pose mit der zweiten Pose, wobei in Abhängigkeit des durchgeführten Vergleichs mittels der Verifizierungseinrichtung ein Konfidenzsignal betreffend die durchgeführte Eigenlokalisierung ausgebbar ist.

According to a first aspect, the object is achieved with a device for evaluating a self-localization of a physical system from sensor data, having:

• - a first determination device for determining a first pose from first sensor data;
• - a second determination device for determining a second pose from second sensor data; and
• - a verification device for verifying the first pose determined by the first determination device, the verification device having:
• A first comparison device for comparing the first pose with the second pose, a confidence signal relating to the self-localization carried out being able to be output by means of the verification device as a function of the comparison carried out.

As a result, improved localization of a physical system from sensor data is advantageously supported. As a result, a statement can be made in an improved manner as to the confidence value with which a localization is carried out. As a result, a variance in the self-localization is smaller, as a result of which trust in the localization solution carried out can be strengthened. In this way, a statement can advantageously be made as to whether a safety criterion relating to localization from sensor data is fulfilled.

1. a) Verwenden von Sensordaten einer ersten Sensormodalität zum Ermitteln einer ersten Pose;
2. b) Verwenden von Sensordaten einer zweiten Sensormodalität zum Ermitteln einer zweiten Pose;
3. c) Vergleichen der in den Schritten a) und b) ermittelten Posen; und
4. d) Ermitteln eines Konfidenzwerts (K) aus dem in Schritt c) durchgeführten Vergleich.

According to a second aspect, the object is achieved with a method for evaluating self-localization using sensor data, having the steps:

1. a) using sensor data of a first sensor modality to determine a first pose;
2. b) using sensor data of a second sensor modality to determine a second pose;
3. c) comparing the poses determined in steps a) and b); and
4. d) determining a confidence value (K) from the comparison carried out in step c).

According to a third aspect, the object is achieved with a computer program product with program code means for carrying out the proposed method when it runs on a proposed electronic device or is stored on a computer-readable data carrier. Advantageous developments of the device and the method are the subject of the respective dependent claims.

A further advantageous development of the device provides that the first pose and the second pose can be determined from sensor data based on different sensing principles. As a result, a variance in the determined poses can advantageously be reduced because they were determined using different sensor modalities.

A further advantageous development of the device provides that the first determination device is set up to determine the first pose from lidar data and/or radar data and/or inertial measurement data and/or steering angle data and/or odometry data and/or satellite-based GNSS data. In this way, a variance of the determined poses relating to self-localization can advantageously be reduced.

A further advantageous development of the device provides that the second determination device is set up to determine the second pose from lidar data and/or radar data and/or inertial measurement data and/or steering angle data and/or odometry data and/or satellite-based GNSS data. In this way, too, a variance of the determined poses can advantageously be reduced the. In this way, too, a variance of the determined poses relating to the self-localization can advantageously be reduced.

A further advantageous development of the device provides that sensor signals are used to determine the first pose by means of the first determination device and for determination of the second pose by means of the second determination device, which sensor signals cause a defined few common cause errors. In this way, sensor data that are best suited to the respective circumstances are advantageously used. For example, for snowfall localization, radar sensor data may be better than lidar sensor data, etc.

A further advantageous development of the device provides that data from a feature layer and threshold values from a threshold data layer can be used from a map database for evaluating the comparison of the poses. In this way, it is advantageously possible to adapt localization accuracies to a concrete, specific environmental situation.

A further advantageous development of the device provides that a status signal relating to a defined localization quality from a feature-based localization can be provided by means of a first comparison device.

A further advantageous development of the device provides that a status signal relating to a defined localization quality based on the compared poses can be provided by means of a second comparison device. In this way, the results of different localization principles are compared with one another, as a result of which the quality of a localization can be determined efficiently.

A further advantageous development of the method provides that the first pose can be validated using a third comparison device from a previous position and a state of the physical system. As a result, e.g. discontinuities that can occur during localization due to regular structures (e.g. avenue trees) can be efficiently eliminated.

An advantageous development of the proposed method provides that when determining the second pose, the first pose is checked for plausibility using sensor data of the same type and/or using sensor data of a different type. In this way, too, a variance in the determined poses relating to self-localization can advantageously be minimized.

A further advantageous development of the proposed method provides that the first and the second pose are determined independently of one another. As a result, the confidence value of the determined first pose (leading pose) can be checked for plausibility even better.

A further advantageous development of the proposed method provides that the confidence value is used to control a self-localized physical system, in particular an at least partially automated vehicle. As a result, a suitable action of the physical system can be initiated in an advantageous manner depending on the confidence value of the localization.
The invention is described in detail below with further features and advantages based on two figures. All of the features described or illustrated form the subject matter of the invention, either alone or in any combination, regardless of how they are summarized in the patent claims or their back-reference, and regardless of their wording or representation in the description or in the figures.

Disclosed features and advantages of the proposed device result in an analogous manner from disclosed features and advantages of the proposed method and vice versa.

• 1 ein prinzipielles Blockschaltbild einer vorgeschlagenen Vorrichtung zum Bewerten einer Eigenlokalisierung mittels Sensordaten; und
• 2 einen prinzipiellen Ablauf einer Ausführungsform des vorgeschlagenen Verfahrens.

In the figures shows:

• 1 a basic block diagram of a proposed device for evaluating a self-localization using sensor data; and
• 2 a basic sequence of an embodiment of the proposed method.

Description of Embodiments

In the following, the term "automated vehicle" is used synonymously with the terms "fully automated vehicle", "autonomous vehicle" and "partially autonomous vehicle".

A core idea of the present invention lies in particular in an evaluation of a self-localization that has been carried out using sensor data, in particular a self-localization of a physical system, for example an at least partially automated vehicle. A confidence or trust value or plausibility value associated with the self-localization is determined and output.

From a security point of view, a trustworthy localization of an at least partially automated physical system is essential. For this purpose, every error must be recognized if possible, to ensure safe operation of the physical system. Based on current risk and hazard analyzes in the field of autonomous mobility, localization usually requires maximum functional safety and integrity.

The proposed method can be seen as a redundant extension of a functional self-localization that is known per se, with the extension taking advantage of the introduction of several additional modalities. A functional localization can be seen as a combination of the localization algorithm and the associated sensors. The aim of the proposed localization plausibility check is in particular to confirm the poses determined from the localizations, with a “pose” being understood below to mean a local position (e.g. in x/y coordinates) and preferably an orientation or alignment of the physical system to be localized . All functions that depend on a pose (e.g. path planning, control, etc.) use the plausible localization pose.

It is conceivable, for example, to validate a lidar-based localization either by additional localization using radar sensors or by GNSS-based localization. A verification device is preferably set up in such a way that a sensor set for providing the sensor data is scalable. In particular, this means that a physical system to be located (e.g. an automated vehicle), which is moving in terrain where the use of GNSS-based localization is not expedient, does not have to be equipped with satellite-based GNSS sensors. For corresponding applications, it is possible without any problems to use both the radar sensor system and the GNSS-based sensor system for the plausibility check.

It is decisive for meeting safety requirements that the localizations compared in the plausibility check are essentially independent of one another, with this principle equally affecting sensors and algorithms. It should therefore be ensured that possible errors do not correlate with one another. Accordingly, different measuring principles should be used on the sensor side.

Both the lidar-based localization and the GNSS-based localization can use an inertial sensor system 1a (inertial measurement unit, IMU), a steering angle sensor 1d, 1h (steering angle sensor, SAS) and a wheel speed sensor 1b, 1f (wheel speed sensor, WSS). Redundant sensors of the types mentioned, i.e. different sensor modalities, are preferably used to determine the first and second poses P1, P2.

The second pose P2 calculated by the GNSS-based or radar-based localization can be referred to as a “cover pose” since it is used to check the first pose P1 (“lead pose”) from the lidar-based localization.

If a difference between the first pose P1 and the second pose P2 exceeds a defined threshold value, the verification device 30 responds and classifies the localization as uncertain or untrustworthy. The data or threshold values mentioned can be stored in a semantic map database 200, for example, and can vary depending on the environment.

The algorithms and sensors used for localization can be arbitrarily replaced, expanded and scaled for the proposed plausibility module.

1 shows a greatly simplified, basic block diagram of a proposed device 100 for evaluating a self-localization using sensor data.

A first determination device 10 can be seen, which processes sensor data from different sensors 1a . . . 1n of a physical unit to be located (e.g. at least partially automated vehicle). The sensor data can be provided by an inertial measuring device 1a, a first wheel speed detection device (odometer) 1b, a lidar device 1c, a first steering angle sensor 1d, etc. while the physical unit is driving. Using the first determination device 10, a first pose P1 of the physical unit to be localized is determined from the sensor data mentioned.

A "Simultaneous Localization and Mapping" (SLAM) algorithm is preferably used for this purpose, with the SLAM algorithm being characterized in particular by the fact that it can process a large number of different sensor data (e.g. radar data, lidar data, camera data, etc.).

Data relating to the ascertained first pose P1 is then fed to a verification device 30, in particular to a first comparison device 31 and to a second comparison device 32 of the verification device 30. Sensor data from a radar sensor 1e is also fed to the first comparison device 31 mentioned.

The sensors 1a . . . 1h can be connected to the device 100 preferably by means of defined interfaces (eg CAN, Ethernet, etc.).

If a comparison of the poses P1, P2 by the second comparison device 32 falls below a defined limit value, the poses P1, P2 are therefore sufficiently similar, the second comparison device 32 outputs a positive status signal S2, otherwise a negative status signal S2.

Sensor data from a second wheel speed detection device 1f, a satellite-based GNSS device 1g and a second steering angle sensor 1h are fed to a second determination device 20, which can have a microcomputer unit, for example. A second pose P2 of the physical system to be located is determined from the supplied sensor data by means of the second determination device 20 . Data relating to the ascertained second pose P2 are supplied to the second comparison device 32 of the verification device 30 .

A map database 200 can also be seen, in which different data of a feature layer 210 and threshold value data of a threshold value layer 220 are stored. In this case, the threshold value data mentioned include threshold values for the accuracy of a localization that has been carried out.

The threshold value data of the threshold value layer 220 are retrieved from the map database 200 and supplied to the first comparison device 31 and the second comparison device 32 . In this case, a user must define an application for a route to be traveled, with the threshold value data being used to specify how accurate a localization that is carried out should be (e.g. approximately 10 cm accurate, approximately 50 cm accurate, etc.). For example, a threshold for alignment may be about 2°, about 5°, and so on. This can be freely defined depending on the requirement. For this purpose, the threshold value data preferably comprise georeferenced data.

Limit values or tolerances of the determined poses P1, P2, which were determined using different sensor modalities, serve as input data for the second comparison device 32.

The first comparison device 31, on the other hand, is based on a different approach than the second comparison device 32. There, features determined by the sensors 1e . . . 1h are compared with data from the feature layer 210 relating to feature recognition. The feature layer 210 data may include georeferenced metadata. For example, a quality of a feature-based localization can then be determined by the first comparison device 31, which is fed with radar sensor data. This can be explained by the fact that metallic things (e.g. crash barriers, traffic mirrors, etc.) can usually be well represented by radar sensor data.

For example, first comparison device 31 outputs a positive first status signal S1 if the first pose P1 determined by sensor devices 1a, 1b, 1c, 1d corresponds to a defined extent with that by sensor devices 1e, 1f, 1g, 1h and/or second pose P2 ascertained using data from feature layer 210 matches, otherwise a negative first status signal S1.

Because the principles of the comparison devices 31, 32 are implemented differently as a result, independently determined status signals S1, S2 of the comparison devices 31, 32 can be compared with one another using an OR element 34, with a status signal S4 being output at the output of the OR element.

In this way it is known at which local positions what should be detected by a radar sensor 1e of the physical system. For example, a lamp post of the feature layer 210 has metadata with a defined georeference in the form of latitude and longitude. The data ascertained by the radar sensors 1e are compared with this data. If this results in a defined assignment, a confidence value regarding the location can be derived from it.

The features of the feature layer 210 are retrieved from the map database 200 using these features together with the threshold data of the threshold layer 220 representing an accuracy.

For example, it can be determined in this way that three characteristics determined are within a required tolerance threshold regarding localization, as a result of which a positive status signal is determined. How many features have to be recognized is a matter of definition.

Threshold value data of the threshold value layer 220 can, for example, define threshold values between differences in the determined poses P1, P2 for different terrains in which the physical system to be localized is moving. For example, for a required confidence value K, a threshold value relating to the differences between the determined poses P1, P2 can be smaller on narrow streets than on wide streets, etc. The threshold value data of threshold value data layer 220 and the Data of the feature layer 210 can, for example, also define requirements for localization accuracy under different weather conditions.

The data mentioned can be used to specify how precisely the physical system should be localized based on external circumstances, the degree to which the physical system should run without interruption, etc.

Data relating to the determined first pose P1 can optionally also be fed to a third comparison device 33, by means of which discontinuities between map data in the map database 200 and sensor data are determined, for example due to regular structures in the area, such as avenue trees, etc.

The third comparison device 33 functions as follows:

A vehicle is in a defined state at all times, which is defined by parameters (e.g. steering angle, speed, acceleration, etc.), and depending on this defined state, the vehicle can cover a defined distance in a specific time interval. For example, a lidar SLAM algorithm is performed with a frequency of 10 Hz, which corresponds to a time interval of 0.1s in which a new localization is performed. For example, in an avenue, scans can also be suitable for another location, because the structural change in the avenue trees is repeated regularly. As a result, it could happen that the lidar algorithm makes jumps in the localization.

The previously determined pose P1 must therefore be stored in the third comparison device 33 . If it is then determined that the newly determined pose P1 is not within a tolerance range (due to the defined vehicle state), the status signal S3 can be used to indicate that this vehicle state is not possible at all. Such discontinuities can advantageously be intercepted by means of the third comparison device 33 .

As a result, the comparison devices 31, 32, 33 can be used to make a statement regarding the localization quality of the physical system. This can be advantageously justified by the modularity of the proposed approach: a positive status signal S4 requires either only a positive status signal S1 (result of the first comparison device 31) or a positive status signal S2 (result of the second comparison device 32).

As a result, this means that three or more sensor modalities are not absolutely necessary for a high localization accuracy, but this can be achieved with two sensor modalities.

Technically, this can be achieved as follows:

The first comparison device 31 determines a first status signal S1 and the second comparison device 32 determines a second status signal S2, with the status signals S1, S2 being fed to a logical OR element 34. A status signal S3 from the third comparison device 33 is fed to a logical AND element 35, to which the result of the logical OR element 34 is also fed.

As a result, a confidence signal K relating to the localization carried out is generated and output by means of the AND element 35 . Suitable measures can be taken downstream of the device 100 based on the confidence signal K (e.g. by means of an electronic control unit), for example a standstill of the self-localized physical system, a defined safe state, etc. can be initiated.

If the confidence signal K=0, this establishes that the self-localization is not usable at the moment. For example, an emergency stop can be initiated or a command can be issued that it is not possible to continue driving with the current localization quality. Advantageously, hazardous situations can be largely avoided in this way.

The sensor data of the sensors 1a . . . 1f are preferably redundant in each case for the first pose P1 to be determined and for determining the second pose P2. The results of said first pose P1 of said pose are preferably based on non-common-cause causes, it being conceivable, for example, that radar data is better suited than lidar data for localization in snow, and satellite-based GNSS data for localization within a desert area are more suitable than radar data, etc.

As a result, poses P1, P2 determined with different sensing principles can be verified in the manner described.

As a result, a vehicle can be controlled at least partially autonomously if the confidence signal is K=1.

It goes without saying that the mentioned sensors 1a .

• - Anstatt die Lokalisierungsgenauigkeit durch die Fusion aller verfügbaren Sensordaten zu optimieren, wird die ausreichend genaue Lösung, die aus einem Teil der Sensoren berechnet wird, durch die Plausibilisierung aussagekräftiger und vertrauenswürdiger
• - Hohe Verfügbarkeit des Systems bei sehr hoher Fehlerintegrität
• - Flexibler Einsatz in unterschiedlichsten Terrains

Due to the proposed plausibility check solution, only the plausibility check algorithm, but not the localization algorithm, has to be implemented “safely” in the manner described. For this purpose, two independent localizations (using sensors and algorithms) must be compared within a plausibility module, which results in the following beneficial effects:

• - Instead of optimizing the localization accuracy through the fusion of all available sensor data, the sufficiently accurate solution calculated from a part of the sensors becomes more meaningful and trustworthy through the plausibility check
• - High availability of the system with very high error integrity
• - Flexible use in different terrains

At least one of the comparison devices 31, 32, 33, preferably all comparison devices 31, 32, 33, can be implemented as a software program or as different software programs in each case. The entire device 100 is preferably installed in the physical system to be localized.

2 shows a basic sequence of a proposed method in a highly schematic manner.

In a step 300, first sensor data is used to determine a first pose P1.

In a step 310, second sensor data is used to determine a second pose P1.

In a step 320, the poses P1, P2 determined in steps a) and b) are compared.

In a step 330, a confidence value K is determined from the comparison carried out in step c).

In summary, an evaluation of a localization (position determination) in a fail-safe system (e.g. vehicle may stop in the event of an error in order to reach a safe state) is carried out with the present invention. By doing without a safety driver, there are high safety requirements for localization, which can be met by the proposed device and the proposed method. Advantageously, two localizations are compared instead of a conventional merging of all sensor data into one localization. As a result, an improvement in the security and trustworthiness of performance-heavy localization solutions is supported.

The method according to the invention can advantageously be implemented as software that runs on the proposed device 100, for example. A simple adaptability of the method is supported in this way. It is conceivable that the software of the device 100 is at least partially cloud-based.

The person skilled in the art will modify and/or combine the features of the invention in a suitable manner without departing from the essence of the invention.

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• DE 102017221691 A1 [0002]
• DE 102015211279 A1 [0003]
• EP 3462134 A1 [0004]

Claims (14)

Device (100) for evaluating a self-localization of a physical system using sensor data, comprising: - A first determination device (10) for determining a first pose (P1) from first sensor data; - A second determination device (20) for determining a second pose (P2) from second sensor data; and - a verification device (30) for verifying the first pose (P1) determined by the first determination device (10), the verification device (30) having: - a first comparison device (31) for comparing the first pose (P1) with the second pose (P2), wherein a confidence signal (K) relating to a confidence in the self-localization carried out can be output by means of the verification device (30) depending on the comparison carried out. Device (100) according to claim 1 , characterized in that the first pose (P1) and the second pose (P2) can be determined from sensor data based on different sensing principles. Device (100) according to claim 1 or 2 , characterized in that the first determination device (10) is set up to determine the first pose (P1) from lidar data and/or radar data and/or inertial measurement data and/or steering angle data and/or odometry data and/or satellite-based GNSS data. Device (100) according to one of the preceding claims, characterized in that the second determination device (20) is set up to determine the second pose (P2) from lidar data and/or radar data and/or inertial measurement data and/or steering angle data and/or odometry data and/or or to determine satellite-based GNSS data. Device (100) according to one of the preceding claims, characterized in that sensor signals are used to determine the first pose (P1) using the first determination device (10) and to determine the second pose (P2) using the second determination device (20). , which define a few common-cause errors. Device (100) according to one of the preceding claims, characterized in that data from a feature layer (210) and threshold values from a threshold value data layer (220) can be used from a map database (200) for evaluating the comparison of the poses (P1, P2). Device (100) according to claim 6 , characterized in that a status signal (S1) relating to a defined localization quality from a feature-based localization can be provided by means of a first comparison device (31). Device (100) according to claim 6 or 7 , characterized in that a status signal (S2) relating to a defined localization quality based on the compared poses (P1, P2) can be provided by means of a second comparison device (32). Device (100) according to one of the preceding claims, characterized in that the first pose (P1) can be validated from a previous position and a state of the physical system by means of a third comparison device (33). Method for evaluating a self-localization from sensor data, comprising the steps: a) using sensor data of a first sensor modality to determine a first pose (P1); b) using sensor data of a second sensor modality to determine a second pose (P1); c) comparing the poses (P1, P2) determined in steps a) and b); and d) determining a confidence value (K) from the comparison carried out in step c). procedure after claim 10 , wherein when the second pose (P2) is determined, the first pose (P1) is checked for plausibility using sensor data of the same type and/or using sensor data of a different type. procedure after claim 10 or 11 , wherein the first and the second pose (P1, P2) are determined independently of one another. Procedure according to one of Claims 10 until 12 , wherein the confidence value (K) is used to control a self-localized physical system, in particular an at least partially automated vehicle. Computer program product with program code means for carrying out the method according to one of Claims 10 until 13 , if it runs on an electronic device (100) for evaluating a self-localization of a physical system using sensor data or is stored on a computer-readable data carrier.

Images