DE102016215314A1 - Driver assistance system, means of transportation and method for predicting a traffic situation - Google Patents

Abstract

A driver assistance system, a means of locomotion and a method for predicting a traffic situation are proposed. The method comprises the steps of: determining a current image of an environment (30) of a vehicle (10), determining a future state (xk) of an environment object (20) contained in the image by means of a parametric model, determining a correction value (g (g) xk)) by means of a machine-based learning method, - determining an uncertainty value (Pk + 1) by means of the machine-based learning method, and - determining a tolerance band (6) for future states (xk) by means of the correction value (g (xk)) and the uncertainty value (Pk +1) for prediction of the vehicle environment (30).

Description

The present invention relates to a driver assistance system, a means of transportation and a method for predicting a traffic situation. In particular, the present invention relates to improvements in the precision of a predicated vehicle environment.

Vehicles increasingly have environment-analyzing driver assistance systems. In addition, the first driving robots are used in series production, which make large parts of the vehicle control automatically.

In order for an autonomously driving motor vehicle to be able to integrate in the traffic flow, the vehicle must have the positions of the interaction partners in the environment. For example, to enable efficient, collision-free path planning, predictions about the future whereabouts of other road users should also be available. For this the trajectory prediction is used.

In the current state of the art, there are two different approaches to how this prediction is performed. On the one hand, the relevant objects are tracked and with regard to their current physical variables (velocity, acceleration, yaw rate, etc.) a prediction about the future position is made by means of a parametric model (that is, by equations). On the other hand, a learning process (machine learning) compares the behavior of the relevant objects with stored training data, and on the basis of this a prediction is made based on "experience-based".

For the prediction of driving maneuvers already partially probabilistic approaches with support vector machines, (dynamic) Bayesian networks and / or Hidden Markov models are used.

DE 10 2009 057 978 A1 discloses an apparatus and method for predicting the position and / or movement of a road user relative to a vehicle, apparatus for predicting an imminent collision of a road user with a vehicle.

DE 10 2012 009 555 A1 discloses a method of assisting a driver in driving a vehicle and a corresponding driver assistance system.

US 201 1/31 3665 A discloses a method for automatically detecting a driving maneuver of a vehicle and a driver assistance system implementing the method.

WO 14 192 137 A1 discloses an apparatus for predicting a movement trajectory and a corresponding method.

In the aforementioned prior art, an analytical formula relationship for a behavior is not always known. Learning methods are sometimes based only on a so-called "maximum likelihood estimation" of the future state, but do not use Bayesian description for a probability distribution of the state. Both approaches (physical equations or learning methods) are usually used separately. The algorithms are usually used to specifically answer a specific question, whereby a classification of discrete events (eg "lane change in 3 seconds - yes / no" or probability for hypothesis) takes place.

Based on the aforementioned prior art, it is an object of the present invention to meet the need in the art for improved reliability and precision of prediction.

The above object is achieved by a method for predicting a traffic situation. In other words, according to the invention, the prediction of such a section of a traffic situation, which is management-relevant for an ego means of locomotion / vehicle, takes place. First, a current image of an environment of a vehicle (ego vehicle) is determined by means of a sensor or a plurality of sensors. Such sensors are often referred to as "environmental sensors" and, in addition to the method according to the invention, can also be used for parking / distance assistance applications. Subsequently, a future state of an environment object contained in the image is determined by means of a parametric model. Parametric models use equations which, for example in the form of equations of motion, yield deterministic results for velocity, distance and position of a bypass object. As far as the method known in the prior art with respect to the evaluation of sensor signals is also used according to the invention. In addition, a correction value is determined by means of a machine-based learning method, which is also referred to in the prior art as "machine learning algorithms" (MLA). The correction value represents a supplementary parameter for the prediction of the environment of the vehicle, can draw on experience-based findings and can therefore take into account situational boundary conditions and dependencies of different components in the image of the environment. In addition, an uncertainty value is determined by means of the machine-based learning method. The uncertainty value represents one Indicator of how reliable the prediction is by means of the machine-based learning process. Thus, for example, the uncertainty value can provide information as to which area around a means of transportation has to be regarded as "safely free" or "reliably occupied" on the basis of the prediction. Subsequently, a tolerance band for the future state is determined by means of the correction value and the uncertainty value for the prediction of the vehicle environment. The tolerance band can also be understood as a "probability distribution of the environment object contained in the image". As a result, a combination of parametric analysis of an image of a vehicle environment and knowledge from a machine-based learning process fed with the image of the vehicle environment is proposed. As a result, more accurate predictions of the components of a vehicle environment can be generated based on sensory information.

The dependent claims show preferred developments of the invention.

The inventively determined future state of the current environment object can be determined, for example, by parametrically determining a location and / or a motion variable as a parameter of the environment object. For example, an equation of motion for temporally successive positions of the image of the environment object can be set up and from this the speed and / or the position of the environment object can be determined at a later time. In particular for the recommendation or fully automatic initiation of driving maneuvers, the future position, speed or acceleration of a means of locomotion is highly relevant and therefore can be determined according to the invention advantageously and precisely.

For imaging the vehicle environment, for example, ultrasonic sensors can be used, as has long been known for parking assistance. Alternatively or additionally, optical sensors (eg, a camera and / or laser / lidar systems and / or radar systems) may be used to scan surrounding areas of the ego vehicle and provide corresponding data for use in a method of the invention.

The environment object may be a foreign vehicle, a pedestrian, an animal, or another road user. Alternatively or additionally, stationary environmental objects (eg a guardrail, a tree, a pillar, a building or the like) can also be detected according to the invention and predicted to predict a future state of a mobile object. For example, a lane change to the left becomes very unlikely if a guardrail runs to the left of the vehicle. Likewise, open spaces (also called "negative objects") and / or lane markings (eg lane boundary, guideline, or the like) can be understood as environment objects and predicted with regard to their future state. In this context, it should be noted that the future state of stationary environment objects is particularly advantageous according to the invention if the ego vehicle has a non-deterministically known future position, for example because a driver takes over the guidance of the means of locomotion, instead of a driving robot driving one Trajectory has predicted.

Based on the prediction of the vehicle environment, it is preferably possible to classify a surrounding area (eg, a lane in front of or next to the ego vehicle) to determine whether the surrounding area can be assumed to be "occupied" or "unoccupied" in the future. In other words, it is determined on the basis of the inventively created prediction whether a risk of collision or no risk of collision or a hindrance of other road users or the ego means of locomotion is to be expected for the surrounding area.

In response to the prediction of the traffic situation according to the invention, a longitudinal guidance and / or a transverse guidance of the means of locomotion can be carried out automatically. For example, a driving robot automatically adjust the longitudinal guide and / or the transverse guide and thus ensure a comfortable and safe movement of the vehicle.

To provide a powerful machine-based learning technique, training data may be created during development using a sensor of ego-locomotion (or similar means of locomotion). For example, sensor data for a variety of different traffic situations can be collected to detect and archive interdependencies in the data. For example, free spaces arising in front of a third-party vehicle can be linked with acceleration processes of these vehicles, or vehicles that crash into vehicles from outside vehicles can be linked to the usually occurring deceleration processes of the foreign vehicles. These findings can be stored in data form in a memory of the vehicle. The data can represent the dependencies between the detected events in the image of the environment. Depending on which type of traffic situations are to be predicted by the method according to the invention, the corresponding traffic situations can be modeled, simulated and sensed in the training run in order to increase the experience of the system for the machine-based learning process.

The training data determined as described above can then be read out for determining the correction value and used in the context of the method according to the invention. In particular, it can be recognized during use that no sufficient training data is available for determining a sufficiently narrow tolerance band by means of the correction value. This can be based, for example, on the fact that the current traffic situation was not learned during the training runs or was not recognized sufficiently frequently (eg in each case modified). In response, it may be concluded that the parametric model is used exclusively for predicting the vehicle environment so as not to adversely affect its results with less reliable findings.

According to a second aspect of the present invention, a driver assistance system is proposed, which may be, for example, components of a lane assistant, a distance holding system, a driving robot, or the like. The driver assistance system comprises an environment sensor which is set up to detect a vehicle environment of the vehicle and to provide corresponding data for an evaluation unit. This is set up to determine a future state of an environment object contained in the image (or of several environment objects) or of a surrounding area contained in the image by means of a parametric model (eg for a traffic situation). According to the invention, a correction value and an uncertainty value are determined by means of a machine-based learning method. Subsequently, a tolerance band / a probability distribution for the future state is determined by means of the correction value and the uncertainty value and used for the prediction of the vehicle environment. This use may include, for example, a weighting of the future state of the environment object determined on the basis of the parametric model and the machine-based learned correction value and uncertainty value. In this way, the driver assistance system is set up to realize the features, combinations of features and advantages of the method according to the invention in a corresponding manner, so that reference is made to avoid repetition of the above statements.

According to a third aspect of the present invention, a vehicle is proposed, which has an inventive driver assistance system according to the second-mentioned aspect of the invention. The vehicle may be, for example, a car, a van, a truck, a motorcycle, or the like.

Further details, features and advantages of the invention will become apparent from the following description and the figures. Show it:

1 a schematic representation of a traffic situation in which a method is carried out according to an embodiment of the present invention;

2 an illustration of different future state of objects for surrounding objects, comparing in accordance with the prior art and results determined according to the invention with each other;

3 an illustration of a tolerance band created according to the invention, a training points-based function, which was created by means of a machine-based learning algorithm; and

4 a flowchart illustrating steps of an embodiment of a method according to the invention for the prediction of a traffic situation.

1 shows a car 10 as an ego vehicle, which has a forward-looking radar sensor 1 as a registration unit. This is with an electronic control unit 2 connected as an evaluation unit information technology. The electronic control unit 2 is in turn with a data store 9 which holds results of a training of the machine-based learning method as well as computer program code for carrying out a method according to the invention. In the coverage area 3 of the radar sensor 1 are a guardrail 8th as a fixed environment object and a foreign vehicle 20 as a mobile environment object of a vehicle environment 30 arranged.

2 shows a schematic representation at different times kk + 1, k + 2, k + 3 according to the prior art determined one or more dimensional state x _k , x _{k + 1} , x _{k + 2} , x _{k + 3 of} an environment object 20 and respective associated uncertainty ellipses P _k , P _{k + 1} , P _{k + 2} , P _{k + 3} , which are results of a parametric model for the state prediction generated by means of a Kalman filter, and with increasing prediction time a large increase of the uncertainty ellipses P _k , P _{k + 1} , P _{k + 2} , P _{k + 3} . Through the states x _k , x _{k + 1} , P _{k + 2} , P _{k + 3} a compensation curve T (k) runs. The state prediction results as x _{k + 1} = T (x _k ). with the uncertainty _{Pk + 1} = _Pk + Σf _{, k} .

Compared with the a present invention performed semi-parametric determination of the single- or multi-dimensional condition x _'k x' _{k + 2,} x _{'k + 3,} etc. which with increasing Prädiktionszeitraum from parametrically determined state x _k, x _{k + 1,} x _{k +2} , x _{k + 3 is} different. The semi-parametric state prediction according to the invention results as x ' _{k + 1} = f (x' _k ) + μ _g (x ') wherein the uncertainty determined according to the invention results as follows: P ' _{k + 1} = P' _k + Σ _{g, k} .

From the above relationships it follows that the parametric Zustandsprädiktion f (x _k) is corrected according to the invention with g (x _k). The increase in uncertainty F from the machine-based learning method per time step k is generally less than the increase in uncertainty Σ _{f, k} , which results according to the prior art. This is in 2 can be seen on the basis of the considerably reduced uncertainty ellipses _{P'k + 1} , _{P'k + 2} , _{P'k + 3} , which lead to a compensating curve T '(k) which is changed in comparison with the prior art and has a less tolerance.

3 shows a comparison of the input quantities A and the outputs B of a Gauss process which can be used according to the invention with "Squared-Exponential (SE) = Radial Basis Function (RBF) Kernel". Training points determined in a training session 4 form support points of a machine-based learned function 5 , which by a tolerance band 6 is surrounded. The more nodes in the range of the input sizes A, the narrower the tolerance band 6 , In areas 7 With few or no support points (input quantities in the range -1 or above 10), the tolerance range widens 6 drastically. Here the function value or output B drops back to "zero" and the uncertainty rises to a maximum value. This usually corresponds to Σ _{f, k} . The output value B (in 3 merely one-dimensional, but generally also n-dimensional) corresponds to the correction value for the prediction according to the invention.

4 shows steps of an embodiment of a method according to the invention for the prediction of a traffic situation. In step 100 Data from a vehicle environment are captured and stored. Here, sensor data are recorded and stored. In step 200 the method according to the invention is taught in a training run and recognizes mutual dependencies of the stored data. Optimizations of learned dependencies can be performed at any time. If the results are not satisfactory, you can re-train. In step 300 Dependency data representing the dependencies is stored in a memory of the vehicle. This essentially completes the training run. In step 400 a current image of an environment of a vehicle is determined by means of vehicle-based sensors. In step 500 a future state of an environment object or region contained in the image is determined by means of a parametric model for the environment object. Here, for example, equations of motion are used, which generate essentially exactly one output variable for each input variable. In step 600 a correction value is determined by means of a machine-based learning method. Accordingly, in step 700 determines an uncertainty value by means of the machine-based learning method. In step 800 a tolerance band for the future state is determined by means of the correction value and the uncertainty value for prediction of the vehicle environment. The tolerance band takes into account a presumed or actual precision of the prediction according to the invention. The steps 500 . 600 and 700 can also be executed parallel in time. This way, in step 900 a surrounding area (eg, a lane) is classified as "occupied" or "unoccupied" based on the prediction of the vehicle environment. Subsequently, in step 1000 an automatic adaptation of a longitudinal guide and / or a transverse guide of the vehicle as a function of the prediction of the vehicle environment. In other words, a driving robot is instructed to take over the longitudinal and / or transverse guidance of the vehicle as a function of the prediction of the vehicle environment.

A core idea of the invention comprises the idea to use the equation-based parametric model of a trajectory prediction as a prior knowledge for the predictions and to improve it by a machine learning approach. The improvement of the equation model happens via the utilization of training data. Thus, areas without available training data have an improvement equal to zero and use only the equation model as a fallback level. Areas with training data can reach an improved model. However, the basis still provides the equation model, while the machine learning process provides for improvement, depending on the available training data.

• • Kombination von parametrischen Modellen f(x_k) als a priori Wissen mit maschinellem Lernverfahren, beispielsweise nicht-parametrischen bayes'schen Methoden g(x_k), wie Gauß-Prozesse: x_k+1 = f(x_k) + g(x_k) + ∊ mit aktuellen und prädiziertem Zustand x_k und x_k+1.
• • Das Lernverfahren wird dabei auf systematische Fehler und/oder nicht modellierte Zusammenhänge des parametrischen Modells trainiert: x_k+1 – f(x_k) = g(x_k) + ∊ mit dem Fehler ∊ als additivem, mittelwertfreiem, gaußverteiltem Rauschen ∊ ~ N (multivariate Gauß- bzw. Normalverteilung N).
• • Da das parametrisches Modell dennoch als a priori Wissen verwendet wird, wird in Bereichen des Eingangsraums, in denen keine oder nur wenige Trainingsdaten zur Verfügung standen und das Lernverfahren somit nur unsichere Aussagen treffen kann, automatisch und kontinuierlich auf das parametrische Modell zurückgegriffen. Das Lernverfahren, beispielsweise Gauß-Prozess, gibt als bayes'schen Methode zusätzlich zum Korrekturwert die jeweilige Unsicherheit auf Grundlage der Trainingsdaten aus: p(x_{k +1}|x_k) = N(f(x_k) + μ_g(x), Σ_g(x_k)) mit Erwartungswert und Kovarianz des Gauß-Prozesses μ_g(x_k) respektive Σ_g(x_k).
• • Transformation zyklischer Variablen (evtl. Position im Fahrstreifen): Entweder als zweidimensionale Mannigfaltigkeit im Einheitskreis und euklidischem Abstand bei kleiner Kernel-Bandbreite als Näherung; oder exakt als Winkel im Einheitskreis und Anpassung des Abstandsmaß im Kernel für diese Variable.
• • Ist das dynamische Verhalten durch die semi-parametrische Modellierung ausreichend gut, kann beispielsweise ein Fahrstreifenwechsel auf Grundlage der Zustände des Eingangsraums vorhergesagt werden: Dafür wird iterativ die Wahrscheinlichkeitsdichtefunktion p(x_{k +n}|x_k) nach n Zeitschritten bestimmen, vorzugsweise mit Partikel Filter und N Partikeln als Gaussian Mixture Model p(x_{k +n}|x_k) = Σ N-1 / i=0 w (i) / k+nN (i) / k+n, Σ (i) / k+n mit Gewichtungsfaktor w (i) / k+n des i-ten Partikel zum Zeitpunkt k + n, um multimodale Wahrscheinlichkeitsdichtefunktion für mehrere Fahrstreifenhypothesen zu modellieren. Mit der kumulativen Verteilungsfunktion P_x\y = ∫ b / ap(x_k+n|x_k)dy wird dann die Wahrscheinlichkeit für die gewünschte Hypothese y ∊[a; b] bestimmt. Dabei ist y die Größe des Zustandsvektors x, über die integriert wird. Die Funktion P beschreibt bspw. die Wahrscheinlichkeit, dass das Objekt mit dem Zustand x_k+n zum Zeitpunkt k + n sich in dem Fahrstreifen befindet, welcher in seiner Breite durch das Intervall [a; b] definiert ist. Je nach Wert der errechneten Wahrscheinlichkeit gilt ein Fahrstreifen als „belegt” oder „frei”. Überschreitet die Wahrscheinlichkeit einen festgelegten Schwellenwert, wird der Fahrstreifen als von diesem Objekt „belegt” betrachtet.

The seamless communication between the systems of equations and the learning methods takes place, for example, as follows:

• Combination of parametric models f (x _k ) as a priori knowledge with machine learning, for example non-parametric Bayesian methods g (x _k ), such as Gaussian processes: x _{k + 1} = f (x _k ) + g ( x _k ) + ε with current and predicted state x _k and x _{k + 1} .
• • The learning process is trained on systematic errors and / or non-modeled relationships of the parametric model: x _{k + 1} - f (x _k ) = g (x _k ) + ε with the error ε as additive, mean-free, Gaussian distributed noise ε ~ N (multivariate Gaussian or normal distribution N).
• • Since the parametric model is still used as a priori knowledge, the parametric model is automatically and continuously used in areas of the entry room where no or only a few training data were available and the learning process can only make uncertain statements. The learning method, for example the Gaussian process, as a Bayesian method, in addition to the correction value, outputs the respective uncertainty on the basis of the training data: p (x _{k +1} x x _k ) = N (f (x _k ) + μ _g (x) , Σ _g (x _k )) with expected value and covariance of the Gaussian process μ _g (x _k ) respectively Σ _g (x _k ).
• • Transformation of cyclic variables (possibly position in the lane): Either as a two-dimensional manifold in the unit circle and Euclidean distance with small kernel bandwidth as approximation; or exactly as an angle in the unit circle and adjusting the distance measure in the kernel for that variable.
• If the dynamic behavior is sufficiently good due to the semi-parametric modeling, for example, a lane change can be predicted on the basis of the states of the input space. For this, the probability density function p (x _{k + n} | x _k ) is determined iteratively after n time steps, preferably with particles Filters and N particles as Gaussian Mixture Model p (x _{k + n} | x _k ) = Σ N-1 / i = 0 w (i) / k + n N (i) / k + n, Σ (i) / k + n with weighting factor w (i) / k + n of the i-th particle at time k + n to model multimodal probability density function for multiple lane hypotheses. With the cumulative distribution function P _{x \ y} = ∫ b / ap (x _{k + n} | x _k ) dy Then the probability for the desired hypothesis y ε [a; b] determined. Where y is the size of the state vector x over which to integrate. The function P describes, for example, the probability that the object with the state x _{k + n} at the time k + n is located in the lane, which in its width by the interval [a; b] is defined. Depending on the value of the calculated probability, a lane is considered "occupied" or "free". If the probability exceeds a specified threshold, the lane is considered "occupied" by this object.

Applications for semiparametric trajectory predication include, for example, using the lane relevance detection model and recognizing future targets in an ACC with lane change. The model can therefore serve motor vehicles in highly automated and fully automated driving as well as the driver assistance systems of a manually operated vehicle.

The invention is a robust system for predicting a traffic situation using equations of a parametric model as a fallback level. The method according to the invention is less complex than purely machine learning algorithms and can thereby be executed faster, more robustly and / or with less computing power. Compared to the purely parametric prediction of the traffic situation, better, more precise and more reliable results are delivered.

LIST OF REFERENCE NUMBERS

1

radar sensor

2

electronic control unit

3

detection range

4

training points

5

learned function

6

tolerance band

7

Highest tolerance range

8th

barrier

9

data storage

10

car

20

Foreign Vehicle / Environment object

30

Surroundings

100-1000

steps

A

input variables

B

output sizes

G

correction value

k

time step

P

uncertainty ellipse

x

Status

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102009057978 A1 [0006]
• DE 102012009555 A1 [0007]
• US 2011/313665 A [0008]
• WO 14192137 A1 [0009]

Claims (11)

A method of predicting a traffic situation comprising the steps of: - determining ( 400 ) of a current image of an environment ( 30 ) of a vehicle ( 10 ), - Determine ( 500 ) of a future state (x _k ) of an environment object contained in the image ( 20 ) by means of a parametric model, - determining ( 600 ) of a correction value (g (x _k )) by means of a machine-based learning method, - determining ( 700 ) of an uncertainty value (P _{k + 1} ) by means of the machine-based learning method, and - determining ( 800 ) of a tolerance band ( 6 ) for future states (x _k ) by means of the correction value (g (x _k )) and the uncertainty value (P _{k + 1} ) for the prediction of the vehicle environment ( 30 ). The method of claim 1, wherein - determining ( 500 ) of the future state (x _k ) of the current environment object ( 20 ) comprises determining a parameter, in particular a position and / or motion variable. Method according to claim 1 or 2, wherein - the image of the environment ( 30 ) is determined by means of an ultrasonic sensor and / or a camera and / or radar and / or laser-assisted. Method according to one of the preceding claims, wherein the environment object ( 20 ) - a road user and / or - a stationary environment object ( 8th ) and / or - an open area and / or - a lane marking. Method according to one of the preceding claims further comprising - classifying ( 900 ) of a surrounding area based on the prediction of the vehicle environment as "occupied" or as "unoccupied". Method according to one of the preceding claims further comprising - automatic adjustment ( 1000 ) a longitudinal guide and / or a transverse guide of the vehicle ( 10 ) in response to the prediction of the vehicle environment ( 30 ). Method according to one of the preceding claims further comprising - reading out and using training data ( 4 ) for determining the correction value (μ _g (x _k )) and in particular - recognizing that sufficient training data are available for determining a sufficiently narrow tolerance band by means of the correction value (g (x _k )) and in response to this - exclusively using the parametric Model for the prediction of the vehicle environment ( 30 ). Method according to one of the preceding claims further comprising - detecting ( 100 ) of data of a vehicle environment in a training run, - recognition ( 200 ) of mutual dependencies in the data, - storing ( 300 ) of the data and / or dependencies of dependency data in a memory ( 9 ) of the vehicle ( 10 ) and - using the data and / or dependency data to predict the vehicle environment ( 30 ). Driver assistance system comprising - a sensor ( 1 ), which is set up, a vehicle environment ( 30 ) of the vehicle ( 10 ), and - an evaluation unit ( 2 ), which is set up - a future state (x _k ) of an environment object contained in the image ( 20 ) to determine by means of a parametric model, - to determine a correction value (μ _g (x _k )) by means of a machine-based learning method, - to determine an uncertainty value (P _{k + 1} ) by means of the machine-based learning method, and - a tolerance band ( 6 ) for the future state (x _{k + 1} ) by means of the correction value (μ _g (x _k )) and the uncertainty value (P _{k + 1} ) for the prediction of the vehicle environment ( 30 ) to investigate. Driver assistance system according to claim 9, which is arranged to carry out a method according to one of the preceding claims 1 to 8. Vehicle comprising a driver assistance system according to claim 10.

Images