DE102019004842A1 - Method for operating an at least partially automated vehicle - Google Patents

Abstract

In a method for the at least partially automated operation of a vehicle, situation data of the vehicle are determined by means of an environment sensor and an operating sensor of the vehicle, movement data of another vehicle in the vicinity of the vehicle are determined by means of the environment sensor of the vehicle, based on the movement data and the situation data by a Driver assistance system for several possible driving maneuvers of the other vehicle each determines a probability measure and a prediction uncertainty measure, predicts a driving maneuver of the other vehicle using the probability measure and the prediction uncertainty measure, and depending on the predicted driving maneuver of the other vehicle, determines a driving action for the vehicle and transmits it to a driving action control device .

Description

The invention relates to a method for operating an at least partially automated vehicle, such as a passenger car or a truck. In particular, the invention relates to the automated prediction of a driving maneuver, in particular a lane change or keeping another vehicle (old vehicle) in lane in the vicinity of a vehicle (ego vehicle) equipped with a driver assistance system and operated partially automatically with the aid of the driver assistance system and with the help of the procedure.

DE 10 2016 009762 A1 discloses a control system which calculates an overall probability for a lane change of an old vehicle from a movement-based probability and a traffic situation-based probability for a lane change of the old vehicle. Depending on the overall probability, the system can output a signal in order to carry out a speed adjustment of the ego vehicle or a driving maneuver of the ego vehicle. In other words, the known system anticipates such a lane change based on an overall probability for a lane change of an old vehicle and adapts the travel of the ego vehicle accordingly in advance. The known system sometimes detects lane changes too late and has too high a false detection rate.

Anticipating a lane change of another vehicle is understood to mean that before a lane change takes place, which can be defined by driving over a lane marking or falling below a minimum distance to the lane marking and then driving over the lane marking to an adjacent lane, by observing the other vehicle this lane change is predicted, i.e. before the definition criterion is partially or fully met.

Incorrect detection of lane changes is understood to mean that a lane change for another vehicle is predicted in a forecast period, but the other vehicle does not change lane. Such false identifications can reduce the user's trust in a driver assistance system and its acceptance by the user.

The object of the invention is to overcome the disadvantages of the prior art, in particular to provide a method for operating an at least partially automated vehicle that avoids abrupt driving maneuvers when another vehicle enters the lane of one's own vehicle and the incorrect detection of lane changes.

1. a) Situationsdaten des Fahrzeugs mittels eines Umfeldsensors und eines Betriebssensors des Fahrzeugs bestimmt;
2. b) Bewegungsdaten eines anderen, in der Umgebung des Fahrzeugs befindlichen Fahrzeugs mittels des Umfeldsensors des Fahrzeugs bestimmt,
3. c) anhand der Bewegungsdaten und der Situationsdaten durch ein Fahrerassistenzsystem für mehrere mögliche Fahrmanöver des anderen Fahrzeugs jeweils ein Wahrscheinlichkeitsmaß und ein Vorhersageunsicherheitsmaß bestimmt,
4. d) ein Fahrmanöver des anderen Fahrzeugs unter Verwendung des Wahrscheinlichkeitsmaß und des Vorhersageunsicherheitsmaß prädiziert, und
5. e) abhängig vom prädizierten Fahrmanöver des anderen Fahrzeugs, eine Fahraktion für das Fahrzeug bestimmt und an ein Fahraktionssteuergerät übertragen.

This object is achieved by the method according to the invention for at least partially automated operation of a vehicle. Be accordingly

1. a) determining the situation data of the vehicle by means of an environment sensor and an operating sensor of the vehicle;
2. b) movement data of another vehicle located in the vicinity of the vehicle is determined by means of the vehicle's surroundings sensor,
3. c) on the basis of the movement data and the situation data, a driver assistance system determines a probability measure and a prediction uncertainty measure for several possible driving maneuvers of the other vehicle,
4. d) predicts a driving maneuver of the other vehicle using the probability measure and the prediction uncertainty measure, and
5. e) depending on the predicted driving maneuver of the other vehicle, a driving action for the vehicle is determined and transmitted to a driving action control device.

Driving maneuvers are, in particular, precise and exclusive driving maneuvers, keeping lane, changing lanes to the right and changing lanes to the left. The vehicle, which is operated partially automatically with the method according to the invention, is also referred to as an own vehicle or ego vehicle. This vehicle comprises a driver assistance system that is set up and designed to carry out the method according to the invention and to support or completely take over the driving actions of an occupant of the vehicle who at least temporarily controls the vehicle, that is to say the ego vehicle.

Preferred situation data are, in particular, exclusively the longitudinal distance between one's own vehicle and the other vehicle along the roadway and the longitudinal relative speed along the roadway between one's own vehicle and the other vehicle. Preferred movement data are, in particular, exclusively those that directly physically characterize the movement of the other vehicle with regard to a driver assistance task of one's own vehicle, i.e. longitudinal speed, ie the speed along the roadway and lateral speed of the other vehicle and the lateral distance between one's own and the other vehicle. The situation data and the movement data are preferably exclusively longitudinal and Transverse speeds of the vehicle and the other vehicle, as well as variables that can be derived directly therefrom, such as acceleration, distance, relative speed and / or relative distance.

Another vehicle in the vicinity of the vehicle is to be understood as a vehicle for which movement data can be determined by means of at least one environment sensor, preferably an environment sensor of the own vehicle, i.e. measurements of the movement of the other vehicle, in particular relative to the own vehicle . The environment is limited, for example, by the measuring range of an environment sensor or several environment sensors, such as a camera, lidar, radar and / or ultrasound sensor system or any combination of sensor systems operating by means of sensor data fusion.

The probability measure in particular indicates how likely a particular driving maneuver is given the situation data and movement data currently available. The prediction uncertainty measure indicates the uncertainty with regard to the specific probability. By taking into account the probability measure and the prediction uncertainty measure for all possible driving maneuvers under consideration, a more robust statement about the driving maneuver to be expected of the other vehicle is possible. The situation data and movement data comprise information about a limited time period prior to the point in time at which the probability measure and the prediction uncertainty measure are determined.

In a special embodiment, the probability measure and the prediction uncertainty measure are determined by means of a Gaussian process. A Gaussian process model is used to determine the probability measure and the prediction uncertainty measure. In particular, initially unknown parameters of the Gaussian process are determined in at least one optimization step on the basis of training data sets which are based on situation, movement data and assigned known, or manually pre-classified or tagged driving maneuvers. With the help of the Gaussian process, instead of a simple prediction value with regard to the driving maneuver determination, an average probability for the driving maneuver is given, taking into account a wide variety of statistical and possible variants of the driving maneuver, taking into account currently recorded movement and situation data. A large number of factors, such as the driving style or mood of a driver, influence a respective current driving maneuver of another vehicle. Nevertheless, these factors cannot be modeled from the situation data and / or movement data with sufficient predictive power, so that the statistical view of the underlying data and the resulting potential driving maneuvers is advantageous.

In a particular embodiment, a covariance matrix of the Gaussian process is calculated from forecast values of a neural network. In particular, the prediction values of the neural network are obtained from a training data set of several pairs of movement and situation data sets. The combination of the Gaussian process with a neural network represents a step forward compared to the mere use of a neural network in predicting driving maneuvers, because the prediction uncertainty measure indicates whether this prediction of driving maneuvers is of high quality or not.

In a particular embodiment, parameters of the Gaussian process and / or of the neural network are determined in a common optimization step. Overall, a higher quality of the parameters of the neural network is achieved. In particular, the Gaussian process is modeled in such a way that it includes parameters of the neural network and the Gaussian process.

In one embodiment, the neural network is a feed-forward network. In another, preferred embodiment, the neural network is a feedback neural network, in particular a feedback neural network with gated neurons or long-term short memory neurons. In one embodiment, the neural network is a convolutional neural network. Folding neural networks are usually referred to using the English expression convolutional neural network. The folding neural network is preferably designed only one-dimensional.

In one embodiment, the neural network is trained before a common optimization step by optimizing the Gaussian process for determining basic parameters of the neural network. A basic model for the prediction or classification of situation and movement data in driving maneuvers is provided, the parameters of which are further optimized by subsequent coupling with the Gaussian process, i.e. the creation of an extended model using the output values of the neural network in the Gaussian process will.

In a preferred embodiment, a covariance function of the Gaussian process is a quadratic exponential function. The covariance function preferably includes a noise component. With the help of the noise component, an over-adjustment of the parameters of the Gaussian process is avoided during the parameter optimization.

When determining the degree of probability and the degree of prediction uncertainty, recorded situation data and movement data are included for the respective driving maneuvers. The number of the respective driving maneuvers in training data sets is preferably the same.

In particular, the mean expected value for a particular driving maneuver is calculated as the probability measure and the associated standard deviation is calculated as the prediction uncertainty measure.

In one embodiment, the determination of the movement data includes that movement data are predicted from measured movement data with a driver model for the other vehicle. In particular, the driver model is determined by means of the environment sensors of the own vehicle. The driver model is preferably estimated from movement and / or situation data over a driver model observation period that is a multiple of the observation period for predicting the driving maneuver. The respective, possibly aggressive or defensive driving style of the other vehicle can thus be taken into account.

In one embodiment, the prediction uncertainty measure is used as a parameter when determining the driving action. In particular, one's own vehicle is braked or accelerated. A speed change and / or a braking or acceleration change is preferably set as a function of the prediction uncertainty measure.

In particular, the determination of the driving action includes that a driving trajectory is calculated assuming at least one predicted trajectory of the other vehicle. One of the at least one predicted trajectory of the other vehicle is preferably calculated as an extreme case trajectory. In particular, the extreme case trajectory is calculated depending on the prediction uncertainty measure.

In one embodiment, a lane change is predicted by increasing or decreasing the probability measure for a respective one of the driving maneuvers by the prediction uncertainty measure and comparing the resulting probability measures of the driving maneuvers with one another. For an optimistic prediction of lane changes, in particular the probability measure for the driving maneuver in lane keeping is reduced by the associated forecast uncertainty measure and for the driving maneuvers lane change to the left and lane change to the right, the respective probability measures are increased by the associated forecast uncertainty measures.

In particular, the situation data and the movement data from which the driving maneuver of the other vehicle is predicted are determined over an observation period that lies in the past with respect to the prediction time. In particular, the situation data and movement data are discretely resolved according to a time step. Alternatively or additionally, the observation period extends into the future with regard to the prediction time. In particular, at least one data point is predicted for the situation data and / or the movement data. In particular, the at least one data point is predicted using a driver model. In particular, the observation period in a first operating mode is exclusively in the past with respect to the prediction time and extends into the future in a second operating mode of the prediction time, with a change between the first operating mode and the second operating mode depending on the speed of the own vehicle.

One or more embodiments of the invention have the advantage that a lane change by another vehicle is recognized as early as possible and a lane change for another vehicle is predicted as rarely as possible in a prediction period in which no lane change takes place.

• 1: eine Verkehrssituation, in der das erfindungsgemäße Verfahren zu Einsatz kommen kann;
• 2: einen schematischen Überblick über eine erste Variante des Vorhersagemodells, das beim erfindungsgemäßen Verfahren zum Einsatz kommt;
• 3: einen schematischen Überblick über eine zweite Variante des Vorhersagemodells, das bei einer zweiten Ausgestaltung des erfindungsgemäßen Verfahren zum Einsatz kommt;
• 4a: ein Diagramm, das Bewegungsdaten eines anderen Fahrzeugs zeigt;
• 4b: ein Diagramm, das Vorhersagewerte für Fahrmanöver eines neuronalen Netzes zeigt, wie es als Teil des erfindungsgemäßen Verfahrens zum Einsatz kommen kann;
• 4c: ein Diagramm, das qualitativ den Verlauf des mittleren Erwartungswerts für Fahrmanöver eines anderen Fahrzeugs zeigt;
• 4d: ein Diagramm, das qualitativ den Verlauf der Standardabweichung betreffend die Vorhersage der 4c.

Further features, advantages and properties of the invention are explained on the basis of the description of preferred embodiments of the invention with reference to the figures, which show:

• 1 : a traffic situation in which the method according to the invention can be used;
• 2 : a schematic overview of a first variant of the prediction model that is used in the method according to the invention;
• 3 : a schematic overview of a second variant of the prediction model that is used in a second embodiment of the method according to the invention;
• 4a Fig. 3 is a diagram showing movement data of another vehicle;
• 4b : a diagram showing forecast values for driving maneuvers of a neural network, as it can be used as part of the method according to the invention;
• 4c : a diagram which qualitatively shows the course of the mean expected value for driving maneuvers of another vehicle;
• 4d : a diagram qualitatively showing the course of the standard deviation regarding the prediction of the 4c .

1 shows a traffic situation on a multi-lane road. Your own vehicle 10 is equipped with a driver assistance system that works according to the method according to the invention. The task of the driver assistance system can be, for example, the longitudinal speed control with regard to a safe distance from vehicles driving ahead, and the execution of an automatic emergency braking in the event of an impending collision. In order to avoid abrupt driving actions when executing, for example, a distance / longitudinal speed control by the driver assistance system, it is desirable that the driver assistance system be provided with information as early as possible about expected lane changes of others in the vicinity of the own vehicle 10 moving vehicles are provided. The driver assistance system can provide such information, for example an impending lane change to the right of the other vehicle 22nd In order to end an overtaking maneuver, take this into account early on when determining a driving action. For example, the speed of your own vehicle 10 can already be reduced slightly in order to avoid sudden braking, which would be necessary if the lane change with a small distance to the own vehicle were only recognized at a later point in time.

The driver assistance system of the vehicle 10 includes a camera that looks in the forward direction of travel and a 360 ° radar sensor around the front and side of the vehicle 10 vehicles in the vicinity 22nd , 24 , 26th , 28 and to be able to detect their movements. If one accepts the restriction of the detection area, only one forward-looking radar sensor can be provided as the environment sensor. With sensors looking in the reverse direction, you can also be behind your own vehicle 10 other vehicles located 42 , 46 are recorded. The environment sensors are used to collect movement data from the vehicles in the vicinity 22nd , 24 , 26th , 42 , 46 and, as far as partial concealment allows, for the vehicle 28 detected. Movement data characterize the movement of the respective other vehicles through the longitudinal and transverse speed and the transverse distance to the own vehicle. Using the operating sensors of your own vehicle 10 the vehicle's own speeds in the longitudinal and transverse directions are recorded and situation data relating to the other vehicles, namely relative distances and relative speeds, are determined therefrom.

The method according to the invention is also used for predicting a driving maneuver of another vehicle 22nd describe. The method can nevertheless be used for any number of other vehicles in the vicinity of the own vehicle 10 are executed. The driving maneuvers for the other vehicle are predicted to change lanes to the left, to change lanes to the right or to keep lane. In principle, however, it is possible that, when the method according to the invention is carried out, other driving maneuvers are also predicted for which a probability measure and an uncertainty measure can be predicted from the movement data and the situation data.

With the help of at least one environment sensor of the driver assistance system of the own vehicle 10 Movement data of the other vehicle 22nd and situation data determined. As will be described in more detail below, a probability measure and a prediction uncertainty measure are each determined for the driving maneuvers to be predicted, here changing lanes to the left or right or staying in lane. Both the probability measure and the prediction uncertainty measure are included in the prediction of the driving maneuver. For example, the probability measure is reduced by the associated prediction uncertainty measure before a comparison is made with a probability measure determined for another driving maneuver. The probability measure for the other driving maneuver can also be reduced by the prediction uncertainty measure. With this type of prediction, a particularly conservative prediction is made.

With regard to the efficiency of the method, it is crucial here that only movement variables of the other vehicle are used, i.e. no processing of camera image data of the traffic scene is required for the prediction.

The particular robustness of the method is made possible by taking into account a probability measure and a prediction uncertainty measure, which result from a Gaussian process coupled to a neural network being used to predict the driving maneuvers. The coupled predictive model is shown with reference to the schema in 2 explained.

The prediction model comprises a neural network 200 and a Gaussian process 300 . The starting point for a driving maneuver prediction for another vehicle is movement data of the other vehicle and situation data. The movement data and situation data are discretized in terms of time Data of a predetermined length and form a multi-dimensional feature vector 210 . This becomes a neural network 200 to hand over. The output vector 220 includes the classes of the driving maneuvers to be predicted. The neural network 200 is characterized by its structure and its parameters. Structure is understood to mean both its structure and the selected activation functions. The use of a feedback LSTM network is advantageous for the method according to the invention. The neural network 200 forms the feature vector 210 to an output vector 220 from, with between feature vector 210 and output vector to be predicted 220 there is a non-linear functional relationship that includes several initially unknown parameters. The network 200 predicted output vector 220 is adapted to known driving maneuvers for a set of training data by optimization, the parameters of the neural network being adapted in order to minimize a cost function, which in turn depends on the parameters.

The Gaussian process is determined by an expected value function and a covariance function with respect to the input data of the Gaussian process. The relationship is described statistically by the covariance function and its parameters. Various covariance functions are available for modeling, of which a suitable one is selected during the modeling. An exponentially quadratic function is preferably used here as the covariance function. An exponentially quadratic function to which a noise term is added is particularly advantageous. This results in a more flexible course of the covariance function so that the model does not have to map every data point exactly. The Gaussian process is optimized by means of maximum likelihood estimation. As a result, the parameters of the Gaussian process are optimized in such a way that they explain the training data in the best possible way.

The Gaussian process is specifically based on the respective output vector 220 the training data of the neural network is modeled; in other words, the Gaussian process is connected downstream of the neural network. The Gaussian process 300 is thus determined both by parameters of the Gaussian process itself, i.e. parameters of the covariance function, and by parameters of the neural network. When optimizing the Gaussian process on the basis of training data, both the parameters of the Gaussian process 300 including the parameters of the neural network 200 can be determined and / or optimized.

This optimization is usually carried out in advance with training data. An optimization in order to adapt the parameters can also be carried out during the operation of the own vehicle 10 be carried out again and again, in which from the continuously recorded data by means of an evaluation metric a new.

If the Gaussian process and the neural network have been parameterized, it can be determined with for any movement and situation data what the mean expected value is that these correspond to a certain driving maneuver and how large the standard deviation is with regard to this prediction. It is no longer just classified, but also indicates how certain the system is with regard to the classification. For this purpose, unknown situation and movement data is used with the neural network 200 that has an output vector 220 and the Gaussian process, which outputs the mean expected value and the standard deviation for the driving maneuvers.

From the information that the Gaussian process outputs, i.e. the mean expected value and the standard deviation, both as discrete time series data, it is predicted which driving maneuver is actually present, as by block 400 is shown schematically.

A basic parameterization of the neural network is preferably determined before the Gaussian process including the neural network is optimized. To find the basic parameterization, as in 3 shown, the neural network after the output vector 220 an extra layer 230 which uses a softmax activation function to determine a normalized probability for the driving maneuver.

The 4a to 4b show example data which qualitatively illustrate the advantages of the invention. 4a shows the movement data of the other vehicle 22nd , namely the course of the lateral distance and the lateral speed. The data describe a slight drop in the last third of the diagram, which indicates the reduction in the lateral distance and an increase in the lateral speed.

4b shows the prediction of a neural network in which the output vector 220 , as in 3 shown on a layer 230 with the softmax activation function in order to output standardized probabilities with regard to the three driving maneuvers under consideration, keeping lane and changing lanes to the right or left. Qualitatively, it is only possible to confirm late with the aid of the neural network that a lane change is more likely than keeping in lane.

4c shows the standard deviation for the driving maneuvers, which is determined with the aid of the Gaussian process coupled with the neural network. From this it can be seen at a very early point in time that the statement about the driving maneuver is no longer certain. In this phase, the driver assistance system could take precautionary measures, such as reducing your own speed.

The expected value in 4d sinks relatively steeply for the maneuver in lane keeping (µ _LK ). The expected value for the maneuver lane change to the right (µ _LCR ) exceeds the expected value for keeping in lane significantly earlier. The lane change can therefore be predicted earlier.

List of reference symbols

10

Own vehicle

22, 24, 26, 28

other vehicle in front

42, 46

other vehicle behind

200

neural network

210

Feature vector

220

Output vector

230

layer

300

Gaussian process

400

block

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102016009762 A1 [0002]

Claims (10)

Method for at least partially automated operation of a vehicle, in which a. Situation data of the vehicle are determined by means of an environment sensor of the vehicle and an operating sensor of the vehicle, b. Movement data of another vehicle in the vicinity of the vehicle are determined by means of the vehicle's surroundings sensor, c. Using the movement data and the situation data, a driver assistance system determines a probability measure and a prediction uncertainty measure for several possible driving maneuvers of the other vehicle, d. a driving maneuver of the other vehicle is predicted using the probability measure and the prediction uncertainty measure, and e. depending on the predicted driving maneuver of the other vehicle, a driving action for the vehicle is determined and transmitted to a driving action control device. Procedure according to Claim 1 , the probability measure and the prediction uncertainty measure being determined by means of a Gaussian process. Procedure according to Claim 2 , whereby a covariance matrix of the Gaussian process is calculated from prediction values of a neural network. Procedure according to Claim 3 , whereby parameters of the Gaussian process and / or of the neural network are adapted in a common optimization step. Method according to one of the Claims 3 or 4th wherein the neural network is a feedback neural network, in particular a feedback neural network with gated neurons or long-term short memory neurons. Method according to one of the Claims 3 to 5 , wherein the neural network is a convolutional neural network. Method according to one of the Claims 3 to 6th , the neural network being trained prior to a common optimization step by optimizing the Gaussian process to determine basic parameters of the neural network. Method according to one of the preceding Claims 2 to 7th , where a Gaussian covariance function is a quadratic exponential function. Method according to one of the preceding claims, wherein a mean expected value with respect to several sets of situation data movement data and driving maneuvers is calculated as the probability measure for a set of movement and situation data and the corresponding standard deviation is calculated as the uncertainty measure. Method according to one of the preceding claims, wherein the method steps are carried out for a predetermined number of the nearest other vehicles in the vicinity of the vehicle.

Images