DE102023000578A1 - Method for determining potential, future positions of a potential collision object using an adaptive system, and method for training a corresponding adaptive system - Google Patents

Abstract

The invention relates to a method for determining potential, future positions (2) of a potential collision object (3) with which a vehicle (4) can collide, with the determination of the potential, future positions (2) using an adaptive system (1) takes place, which is trained to determine the potential, future positions (2) using a predetermined time horizon (T), whereby the adaptive system (1) is provided with information about the vehicle (4) and/or information about the collision object (3). , and the determination of the potential, future positions (2) based on the information of the vehicle (4) and / or the information of the collision object (3). driver assistance system (12).

Description

The invention relates to a method for determining potential, future positions of a potential collision object with which a vehicle can collide.

Furthermore, the invention relates to a method for training an adaptive system for predicting potential, future positions of a potential collision object with which a vehicle can collide.

The invention also relates to a driver assistance system with an adaptive system.

In the case of an autonomously driving vehicle, such as an at least partially autonomously operated vehicle or a fully autonomously operated vehicle, avoiding collisions with other road users is important. This requires an estimate of where road users will be going. In particular, the movement of people is very dynamic. There are often multiple movement patterns that would be possible and to which the vehicle, particularly the vehicle systems, should respond.

This also makes it difficult to determine at what point in time a potential collision could occur and how far into the future one has to predict this. For example, systems are known that do not estimate probabilistically or only a fixed number of possible trajectories. As a result, however, other possible trajectories are filtered and cannot be used to carry out collision avoidance at an early stage.

Other probabilistic systems use either discretizations or approximated probability distributions, such as "Gaussian Mixtures", for their representation, each of which has a disadvantage in terms of accuracy and/or computing time. In addition, it is estimated for a fixed point in time in the future.

Furthermore, it is disadvantageous in known systems that fixed prediction horizons are used, which makes these systems susceptible to, in particular dynamic, parameters, hyperparameters. Other systems approximate the distribution either by discretizations or simplified probability distributions, which can lead to loss of information. One object of the present invention is to improve improved collision avoidance with dynamic road users, in particular pedestrians.

This object is achieved by a method and a driver assistance system according to the independent patent claims. Useful further developments result from the dependent patent claims.

• - das Ermitteln der potentiellen, zukünftigen Positionen durch ein lernfähiges System erfolgt, das darauf trainiert ist, anhand eines vorgegebenen Zeithorizonts die potentiellen, zukünftigen Positionen zu ermitteln, wobei
• - dem lernfähigen System Informationen des Fahrzeugs und/oder Informationen des Kollisionsobjekts bereitgestellt werden, und
• - das Ermitteln der potentiellen, zukünftigen Positionen anhand den Informationen des Fahrzeugs und/oder den Informationen des Kollisionsobjekts erfolgt.

One aspect of the invention relates to a method for determining potential, future positions of a potential collision object with which a vehicle can collide, with

• - The determination of the potential, future positions is carried out by an adaptive system that is trained to determine the potential, future positions using a predetermined time horizon, wherein
• - Information about the vehicle and/or information about the collision object is made available to the adaptive system, and
• - The potential, future positions are determined using the information from the vehicle and/or the information from the collision object.

The proposed method can improve collision avoidance with dynamic, in particular very dynamic, road users such as pedestrians. This is advantageous above all for partially autonomously or fully autonomously operated vehicles and/or for highly automated vehicles. This is also used for common driver assistance systems in vehicles.

By determining potential, future positions of a potential collision object or multiple potential collision objects, a potential collision between the vehicle and the collision object can be reacted to in advance or at an early stage. In other words, a prediction is made as to the position or positions at which the collision object may be located in the future. This preliminary assessment or prediction of such possible or potential positions of the collision object allows the vehicle and in particular the vehicle's systems to react to them at an early stage, for example to be able to prevent a collision with the collision object.

For example, such a collision object can be a pedestrian or a road user.

For example, the determination of the potential, future positions, ie predicted positions in the future, can be carried out while the vehicle is driving. This can be, for example, in a computing unit of the vehicle or in a computing unit external to the vehicle, such as a cloud application or a backend.

These potential positions can be determined by the adaptive system, such as a neural network and/or a machine learning unit. To determine the potential, future, ie in the future, possible positions of the collision object, the most varied and extensive information can be used as input variables, in particular for the adaptive system. Above all, information about the vehicle and/or information about the collision object can be taken into account. The relationship between the vehicle and the collision object is of particular importance here, since above all a movement of the vehicle relative to a movement of the collision object can be taken into account when determining the potential positions.

It is also conceivable that the determination of the potential, future positions is simulated in a computing unit, and that the real data of the vehicle and/or the collision object are taken into account as input in such a simulation.

In particular, the proposed method can be a computer-implemented method or a cloud-based method or a simulation-based method.

• - Bereitstellen von Informationen des Fahrzeugs und/oder von Informationen des Kollisionsobjekts, und
• - Lernen der Prädiktion der potentiellen, zukünftigen Position des Kollisionsobjekts auf Basis der Informationen des Fahrzeugs und/oder der Informationen des Kollisionsobjekts für einen vorgegebenen Zeithorizont.

A further aspect of the invention relates to a method for training an adaptive system for predicting potential, future positions of a potential collision object with which a vehicle can collide, comprising:

• - Providing information from the vehicle and/or information from the collision object, and
• - Learning to predict the potential, future position of the collision object based on the information from the vehicle and/or the information from the collision object for a predetermined time horizon.

By training the adaptive system, such as a neural network or a machine learning unit, an improved prediction of potential, future positions or whereabouts of collision objects, especially in the area surrounding the vehicle, can be carried out using a wide variety of information, input variables and/or parameters . Consequently, machine or automated learning can be carried out on the basis of the information provided by the vehicle and/or the collision object and/or traffic information and/or map information and/or information about the vehicle’s surroundings, so that future positions or whereabouts of collision objects, in particular dynamically changeable ones, are more efficient and better predicted, i.e. predicted. Such a trained, adaptive system can be used in vehicles, in particular in vehicles that are operated at least partially or fully autonomously, to make an improved, in particular more precise, prediction of potential collisions in order to be able to use this knowledge to carry out improved collision avoidance.

In particular, the method just proposed is a computer-implemented method.

In particular, advantageous embodiments of the further aspect just described can be used as advantageous developments of the aspect mentioned at the outset. In particular, the trained, adaptive system of the further aspect just described can be used advantageously when determining the potential, future position according to the aspect mentioned at the outset.

A further aspect of the invention relates to a driver assistance system with a system capable of learning, wherein the driver assistance system, in particular an electronic driver assistance system, is designed to carry out a method according to one of the preceding aspects or a corresponding embodiment thereof.

In particular, a method according to one of the preceding aspects or an advantageous development thereof can be implemented with the aid of the driver assistance system.

Such a driver assistance system can primarily support the driver of the vehicle and/or an autonomous system of the vehicle to avoid collisions with potential collision objects, such as a pedestrian. Furthermore, at least one potential, future position of a collision object can be determined or predicted autonomously or automatically with the driver assistance system. In this way, potential collisions can be detected at an early stage and appropriate countermeasures can be taken.

For example, the driver assistance system can be further developed according to the possible variations and advantages of the methods described above. The mentioned procedural features male represent corresponding functional features of the driver assistance system. Furthermore, the driver assistance system with the adaptive system has means or functional features with which the methods mentioned can be carried out or executed according to the preceding aspects.

In particular, advantageous embodiments of one aspect can be viewed as advantageous embodiments of the other aspects and vice versa.

Here and below, an artificial neural network, in particular the adaptive system, can be understood as software code that is stored on a computer-readable storage medium and represents one or more networked artificial neurons or can emulate their function. The software code can also contain a number of software code components, which can have different functions, for example. In particular, an artificial neural network can implement a nonlinear model or algorithm that maps an input to an output, where the input is given by an input feature vector or an input sequence and the output is, for example, an output category for a classification task, one or more may include predicted values or a predicted sequence.

Further advantages, features and details of the invention result from the following description of preferred exemplary embodiments and from the drawing(s). The features and combinations of features mentioned above in the description and the features and combinations of features mentioned below in the description of the figures and/or shown alone in the figures can be used not only in the combination specified in each case, but also in other combinations or on their own, without going beyond the scope of the leave invention.

• 1 eine schematische Darstellung eines lernfähigen Systems zur Ermittlung von potentiellen, zukünftigen Positionen eines Kollisionsobjekts;
• 2 eine weitere schematische Darstellung eines lernfähigen Systems aus 1 zur Ermittlung von potentiellen, zukünftigen Positionen eines Kollisionsobjekts; und
• 3 eine schematische Darstellung eines Fahrzeugs, welches mit einem Kollisionsobjekt kollidieren kann.

The following figures show in:

• 1 a schematic representation of an adaptive system for determining potential, future positions of a collision object;
• 2 Another schematic representation of a learning system 1 for determining potential, future positions of a collision object; and
• 3 a schematic representation of a vehicle which can collide with a collision object.

Elements with the same function are provided with the same reference symbols in the figures.

In the 1 an adaptive system 1 is shown, for example. The system 1 capable of learning can be a neural network, in particular an artificial neural network or a network. The system 1 capable of learning can therefore be a software code, for example, which can be stored on a computer-readable storage medium. It is also conceivable that the system 1 capable of learning is a cloud-based unit, for example the system 1 capable of learning can be integrated or implemented in a computing unit, server unit, backend or computer application.

For example, the adaptive system 1 can be used to determine potential, future positions 2 of a potential collision object 3 (cf 3 ) to determine or to determine. In this case, the positions 2 can be determined by the adaptive system 1, which can be trained to determine the positions 2 using a predetermined time horizon T.

The collision object 3 can be a road user, such as a pedestrian. With the help of the adaptive system 1 it can be determined, in particular predicted, at which future position or at which future whereabouts this collision object 3 is located, whereby it can be determined or assessed whether a collision of a vehicle 4 with the collision object 3 is likely or not , in particular how high the probability of a potential collision is.

Thus, with the help of the adaptive system 1, future probabilities and/or future locations, such as the positions 2, of pedestrians, ie the collision object 3, can be predicted. This approach can also be used for other object classes, which, however, behave less complex in their movement pattern compared to a pedestrian.

The given time horizon T is to be understood as a time limit up to which one is likely to plan. In other words, a time horizon is a period of time that one considers, for which one can plan.

In the case of the time horizon T according to the invention, a time horizon that is not too long is to be understood, such as a time horizon of a few seconds or a few minutes. Theoretically, for a very long time rizont can be estimated with very small time steps for each road user and in a very close-knit discrepancy. However, this requires a large computational effort and is difficult to calculate in real time.

In particular, information about the vehicle 4 and/or information about the collision object 3 and/or environmental information and/or traffic information and/or map information can be made available to the adaptive system 1 for determining the positions 2 . For example, a current position 6 of the collision object 3 and/or at least a past position of the collision object 3 and/or a current trajectory 7 (cf 3 ) and/or a past trajectory (compare 3 ) of the collision object 3 are provided. With the information from the vehicle 4, a current position 9 (cf 3 ) of the vehicle 4 and/or at least one past position of the vehicle 4 and/or a current trajectory 10 (cf 3 ) of the vehicle 4 and/or a past trajectory (cf 3 ) of the vehicle 4 are provided to the adaptive system.

An adaptive system can thus be trained by the extensive information, in particular to be able to predict potential positions 2 of the collision object 3 efficiently and in an improved manner in order to improve collision avoidance, in particular interaction with road traffic. Consequently, the interpretability of the system 1 can be increased and less computing power can be determined at the same time. This makes it possible to design or implement a simpler overall system that can also be used in situations that are difficult to predict, such as when it is unclear whether a pedestrian will cross the street or not.

For example, the system capable of learning can be a neural network. Furthermore, it is conceivable that the adaptive system 1 can be integrated into a driver assistance system 12 (cf 3 ) is integrated or implemented. Thus, the prediction or the determination of the potential, future positions 2 of the collision object 3 can already be carried out in the vehicle 4, for example. It is also conceivable that the calculation or determination of these positions 2 takes place externally to the vehicle 1 and is in turn made available via technical communication connections.

An embedding vector 13 can be used in order to be able to represent, in particular determine, information about all possible positions 2 at all possible points in time T1, T2 of the specified time horizon T. For example, embedding (“embedded”) is to be understood as meaning a representation, in particular an intermediate representation, which a neural network, such as the adaptive system 1, has found in order to solve a problem for which it is being trained. This embedding vector 13, which can be referred to as a vector presentation, for example, can be used to represent probabilistic positions, ie positions 2, for any time step, such as times T1, T2, in just one vector. This reduces computing power in particular. A new representation or new type of representation can thus be provided for a trajectory.

In particular, a probabilistic representation and simultaneous interpolation and extrapolation over time for the estimation of positions 2 can be achieved or carried out with the aid of the adaptive system 1 . Furthermore, a representation of estimates integrated probabilistically over time can be passed on to other algorithms based on them, in particular for collision avoidance, without loss of information and with little computational effort. In particular, this embedding vector 13 can be used to output or display the probability distribution of all possible positions 2 for any time steps or the times T1, T2. Here is in the 1 shown by way of example that two possible positions 2 could be determined at time T1. At time T2, a large number of positions 2 have again been determined. In order to be able to display such a visualization of the possible positions 2 , in particular as an image, the adaptive system 1 can perform “heat map decoding” 14 . A better representation can be achieved with such a decoding. Such a visualization can be used, for example, to make the decisions of the network, ie the adaptive system 1, explainable. Various representations can be output, such as a "heat map" for individual time steps T1, T2. Alternatively, however, a possible position at a fixed point in time can also be determined several times from the probability distribution.

In particular, a derivation from the embedding vector 13 can be achieved with the aid of the decoding, which helps to visualize and understand the results. Here, the selection of the parameters of the "heat map" is to be evaluated or defined. However, if the system based on this, such as for collision avoidance (driver assistance system 12), is also a method that relies on machine learning, information can be passed on in a compact form without loss of information by directly using the embedding vector 13 . The building system does not have to find a suitable representation itself to process information. This in turn saves complexity and computing time. For this purpose, information regarding the collision avoidance 16 can be provided in a “down stream task” 15 as a collision prediction or collision avoidance as an output variable. In particular, these can be made available to the driver assistance system 12 or other systems of the vehicle 4 .

In particular, a large number of positions 2 can be determined for a respective point in time T1, T2 of the time horizon T. A probability distribution of the plurality of positions 2 can be determined for the respective point in time T1, T2.

In the 2 For example, the embedding vector 13 can be generated by training the adaptive system 1. Information can be made available to the adaptive system 1 at various points. Here, in a first part, information such as information regarding the collision object 3 and/or the vehicle 1 can be made available or made available as an input 18 to an encoder 17 . The output of the encoder 17, ie the embedding vector 13, can be given in a decoder 19 together with the time horizon T or prediction horizon. Random parameters 20 can also be taken into account here. With the help of the decoder 19, possible positions or positions 2 in the future for a time step, such as the times T1, T2, can be predicted. Here again the positions 2 can be represented as a probability distribution 21 . This probability distribution 21 can be represented in any dimensions. Thus, the method can be presented in two dimensions, such as "Bird Eye View" or as a three-dimensional representation or higher.

By entering, for example, different prediction horizons or the time horizon T, the adaptive system 1 can be trained better, especially during training or learning. The adaptive system 1 thus learns to abstract these different times and allows, for example, an interpolation or an extrapolation. During the interpolation, the training data can be discretized in steps of whole seconds using prediction horizons, for example. The embedding vector 14 or embedding can also be queried for intermediate steps such as 1.3 seconds. When extrapolating, the longest detection horizon in the training data is 5 seconds, for example. However, the embedding 14 can also be queried for a longer prediction horizon or time horizon T, such as 7 seconds, for example.

There are various possibilities for the design of the decoder 19, depending on which representation is more suitable for the given problem. In addition or instead, the embedding 13 can be made available to the head map decoding 14 so that a heat map result 22 can be made available or transmitted. For example, a fixed number of possible positions 2 for the time horizon T can be provided or drawn using the decoder 19 . For this purpose, the decoder 19 can be run through multiple times and the random parameter 20 is added with each run, such as by “Monte Carlo dropout”. The cost term that can be used to learn system 1 is generated by the distance of the best prediction, which would be the “argmax” for the heat map result 22, for example, and the “nearest neighbor” for “sampling”. For example, an actual position can be specified or determined. The position 2 that is closest to the actual position 5 of the collision object 3 can thus be determined or established.

For example, as in the 2 and at least partially in the 1 shown, the adaptive system 1 has an “encoder-decoder architecture”. The encoder can be an encoder and the decoder can be a decoder. For example, an input sequence, ie the input parameter, can be converted into a vector presentation by a so-called embedding shift, for example the embedding vector 13 .

The present invention allows a trajectory to be used as embedding, for example. Thus, probabilistic positions can be determined for any time step. This enables interpolation and extrapolation of the prediction horizon. This means that even very complex probability distributions can be modeled. This enables a compact and quick to process presentation for building systems, especially for collision avoidance. In particular, the adaptive system 1 uses the encoder-decoder architecture of neural networks.

In particular, for determining and in particular for predicting the potential, Future positions 2 simulations, in particular simulation models, and in particular training data sets of the adaptive system 1 are used.

Claims (8)

Method for determining potential, future positions (2) of a potential collision object (3) with which a vehicle (4) can collide, characterized in that - the determination of the potential, future positions (2) by an adaptive system (1) takes place, which is trained to determine the potential, future positions (2) on the basis of a predetermined time horizon (T), the adaptive system (1) being provided with information about the vehicle (4) and/or information about the collision object (3). , and - the potential, future positions (2) are determined using the information from the vehicle (4) and/or the information from the collision object (3). procedure after claim 1 , characterized in that - with the information of the collision object (3) a current position (6) of the collision object (3), and/or at least one past position of the collision object (3), and/or a current trajectory (7) of the collision object (3), and/or a past trajectory (8) of the collision object (3) is provided, and/or - with the information from the vehicle (4), a current position (9) of the vehicle (4), and/or at least one past position of the vehicle (4), and/or a current trajectory (10) of the vehicle (4), and/or a past trajectory (11) of the vehicle (4) is provided. procedure after claim 1 or 2 , characterized in that a large number of potential, future positions (2) of the potential collision object (3) is determined for a respective point in time (T1, T2) of the time horizon (T). procedure after claim 3 , characterized in that a probability distribution (21) of the plurality of potential, future positions (2) of the potential collision object (3) is determined for the respective point in time (T1, T2) of the time horizon (T). Method for training an adaptive system (1) for the prediction of potential, future positions (2) of a potential collision object (3) with which a vehicle (4) can collide, characterized by - Providing information from the vehicle (4) and/or information from the collision object (3), - Learning to predict the potential, future position (2) of the collision object (3) based on the information from the vehicle (4) and/or the information from the collision object (3) for a predetermined time horizon (T). procedure after claim 5 , characterized in that a large number of potential, future positions (2) of the collision object (3) is predicted for a respective point in time (T1, T2) of the time horizon (T), and this is taken into account in the learning. Method according to one of the preceding claims, characterized in that the adaptive system (1) is based on a neural network. Driver assistance system (12) with a learning system (1), wherein the driver assistance system (12) is designed to carry out a method according to one of the preceding claims.

Images