DE102022004427A1 - Method for selecting a trajectory set from a given training data set - Google Patents

Abstract

The invention relates to a method for selecting a trajectory set (S) from a predetermined training data set (D), the selection being made depending on a metric and trajectories (Tj) being iteratively selected from the entire training data set (D) until at least one termination criterion is met .

Description

The invention relates to a method for selecting a trajectory set from a given training data set.

The prediction of movements of road users participating in road traffic, for example vehicles, is an essential part of the operation of automated, for example highly automated, autonomous vehicles. Reliable and safe movement planning is only possible if the prediction is of high quality.

From “Tung Phan-Minh, Elena Corina Grigore, Freddy A. Boulton, Oscar Beijbom, Eric M. Wolff: CoverNet: Multimodal Behavior Prediction using Trajectory Sets; arXiv:1911.10298” is a method for multimodal, probabilistic trajectory prediction for inner-city traffic. The trajectory prediction is carried out by means of a classification using a trajectory set with different trajectories. The size of the trajectory set is limited by a number of different driving measures that can be taken over a predetermined prediction horizon. To generate this trajectory set, each individual trajectory from the trajectory or training data set is considered and it is calculated how many other trajectories in the training data set are covered by it. The resulting degree of coverage is determined by a metric that has a manual hyperparameter. The trajectory that covers most of the other trajectories in the training data set is included in the trajectory set. The process is then continued with all previously uncovered trajectories. It stops when all trajectories are covered. A hyperparameter is used for the degree of coverage, which at least indirectly determines a predefined number of trajectories. The determination of the trajectory set is implemented using a so-called “greedy clustering algorithm”.

Furthermore, “Yuning Chai, Benjamin Sapp, Mayank Bansal, Dragomir Anguelov - MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction; arXiv:1910.05449” is a method for planning the movements of an autonomous vehicle, in which human behavior is predicted. A so-called “K-means clustering” is used to create so-called anchor trajectory sets. A square distance between two trajectories is used to form a cluster.

The invention is based on the object of specifying a novel method for selecting a trajectory set from a given training data set.

The object is achieved according to the invention by a method which has the features specified in claim 1.

Advantageous configurations of the invention are the subject of the subclaims.

• - in einem ersten Verfahrensschritt ein initiales und leeres Trajektorienset angelegt wird,
• - in einem zweiten Verfahrensschritt nacheinander für jedes in dem Trainingsdatensatz vorhandene, aus jeweils zwei Trajektorien des Trainingsdatensatzes bestehende Trajektorienpaar ein Metrikwert gebildet wird, welcher ein Maß angibt, in welchem Umfang eine Trajektorienprädiktion einem Ground-Truth entspricht, und der jeweils gebildete Metrikwert in eine Metrikmatrix eingetragen wird, wobei jedes Element der Metrikmatrix dem jeweiligen Trajektorienpaar zugeordnet ist,
• - in einem dritten Verfahrensschritt ein Metrikspeicher des jeweils aktuellen Trajektoriensets angelegt und mit hohen Werten initialisiert wird, wobei eine Anzahl von in dem Metrikspeicher enthaltenen Elementen einer Anzahl der Trajektorien des Trainingsdatensatzes entspricht,
• - in einem vierten Verfahrensschritt überprüft wird, ob zumindest ein Abbruchkriterium erfüllt ist und
  • - bei Vorliegen des zumindest einen Abbruchkriteriums ein vorliegendes Trajektorienset als metrikoptimales Trajektorienset ausgewählt wird und
  • - bei Nicht-Vorliegen des zumindest einen Abbruchkriteriums ein fünfter Verfahrensschritt solange wiederholt wird, bis zumindest ein Abbruchkriterium erfüllt ist, und
• - in dem fünften Verfahrensschritt eine nächste optimale Trajektorie aus dem Trainingsdatensatz bestimmt wird, wobei
  • - eine temporäre Speicherliste angelegt und initialisiert wird,
  • - über alle Trajektorien in dem Trainingsdatensatz iteriert wird und für jede Trajektorie jeweils ein Minimum in dem Metrikspeicher und der Metrikmatrix ermittelt wird,
  • - ein Mittelwert aller ermittelten Minima gebildet wird,
  • - aus dem Mittelwert ein mittlerer Fehler eines aktuellen Trajektoriensets in Kombination mit einer Trajektorie einer aktuellen Iteration ermittelt wird,
  • - ein Eintrag in der temporären Speicherliste bestimmt wird, welcher den geringsten Mittelwert aufweist,
  • - der Metrikspeicher durch das ermittelte Minimum des Metrikspeichers und der Metrikmatrix ersetzt wird und
  • - die Trajektorie, welcher das ermittelte Minimum zugeordnet ist, in das Trajektorienset aufgenommen wird.

The method for selecting a trajectory set from a given training data set is characterized according to the invention in that

• - in a first process step, an initial and empty trajectory set is created,
• - In a second method step, a metric value is formed successively for each pair of trajectories present in the training data set, each consisting of two trajectories of the training data set, which indicates a measure to which extent a trajectory prediction corresponds to a ground truth, and the metric value formed in each case into a metric matrix is entered, whereby each element of the metric matrix is assigned to the respective pair of trajectories,
• - in a third method step, a metric memory of the current trajectory set is created and initialized with high values, a number of elements contained in the metric memory corresponding to a number of trajectories of the training data set,
• - In a fourth step of the process, it is checked whether at least one termination criterion is met and
  • - if the at least one termination criterion is present, an existing trajectory set is selected as a metric-optimal trajectory set and
  • - If the at least one termination criterion is not present, a fifth process step is repeated until at least one termination criterion is met, and
• - In the fifth method step, a next optimal trajectory is determined from the training data set, where
  • - a temporary storage list is created and initialized,
  • - iterating over all trajectories in the training data set and determining a minimum in the metric memory and the metric matrix for each trajectory,
  • - an average of all determined minima is formed,
  • - a mean error of a current trajectory set is determined from the average in combination with a trajectory of a current iteration,
  • - an entry is determined in the temporary storage list which has the lowest average value,
  • - the metric memory is replaced by the determined minimum of the metric memory and the metric matrix and
  • - the trajectory to which the determined minimum is assigned is included in the trajectory set.

The present method enables, in a particularly advantageous manner, a determination and selection of an optimal trajectory set based on data processing, in particular through comparison, averaging and/or buffering, so that the use of a suboptimal "greedy clustering algorithm" can be omitted and no hyperparameters, for example a degree of coverage, are required. The use of “K-means clustering”, which is also not aimed at enabling downstream prediction models to achieve the best possible prediction quality, can also be omitted. The metric for generating the trajectory set can be selected arbitrarily. For example, this can be selected as Average Displacement Error, ADE for short. The ADE can also be used to evaluate the prediction quality of trajectory prediction models. The generation of the trajectory set is therefore based on the optimization of the prediction quality.

This means that the method enables the generation of a fixed set of trajectories, which is based directly on any or an optionally weighted combination of several metrics. Metrics can also be selected that are used directly to measure the prediction quality. This means that the generation of the trajectory set is already designed to achieve the best possible prediction quality. The method therefore makes it possible to select an optimized trajectory set by means of which a model for predicting the movement of a road user is trained.

Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

• 1 schematisch ein Trajektorienset mit einer Vielzahl von Trajektorien und
• 2 schematisch einen Ablauf eines Verfahrens zur Auswahl eines Trajektoriensets aus einem vorgegebenen Trainingsdatensatz.

Show:

• 1 schematically a trajectory set with a variety of trajectories and
• 2 schematically a sequence of a method for selecting a trajectory set from a given training data set.

Corresponding parts are provided with the same reference numbers in all figures.

In 1 a trajectory set S is shown with a large number of trajectories T _j .

2 shows a sequence of a method for selecting such a trajectory set S from a given training data set D.

The prediction of movements of road users participating in road traffic, for example vehicles, is an essential part of the operation of automated, for example highly automated or autonomous vehicles. Reliable and safe movement planning is only possible if the prediction is of high quality.

There are learning-based methods for trajectory prediction for such a prediction. One type of such learning-based prediction approaches are set-based prediction approaches with a fixed or dynamic trajectory set S.

In classic regression approaches, a training data set D for trajectory prediction is used to learn what a trajectory T _j of the future will look like. A direct output of the learning-based model are coordinates x, y over several time steps, usually in a local coordinate system. In order to achieve multimodality, i.e. to be able to predict a road user, also known as an agent, with several possible futures, a number of prediction heads must be defined before training the model.

The set-based prediction approaches mentioned, on the other hand, have a different functional principle in which, in the case of fixed trajectory sets S, a trajectory set S is extracted from the training data set D using an algorithm before the learning-based model is trained. The trajectory set S consists of some trajectories T _j that may be driven in the future and selected using the algorithm. During training of the learning-based model, the model then learns which trajectory T _j of the trajectory set S most likely corresponds to the future of the agent to be predicted. The regression problem of entire trajectories T _j thus becomes a classification problem transferred. An advantage here is that a number of modes do not have to be determined during training. Even during execution, it can be decided dynamically how many trajectories T _j are selected for further processing, because each trajectory T _j in the trajectory set S is assigned a probability.

During application, this specifically means that trajectories T _j previously extracted from the training data set D and forming the trajectory set S are “selected” for the prediction. This selection is done via a classification using the trajectory set S by determining how likely each individual trajectory T _j of the trajectory set S is to occur in a current traffic scene.

In the present method it is assumed that a training data set D with driven trajectories T _j of an agent trained as a road user is available. It is assumed that the training data set D k contains trajectories T _j , that is, j = 1... k. The trajectories T _j have been recorded by a variety of agents, for example vehicles, during a variety of previous trips. The training data set D is further used for training learning-based prediction models.

In the method, an initial and empty trajectory set S is created in a first method step V1.

wobei R den Zahlenraum der reellen Zahlen repräsentiert und kxk die Dimension der Metrikmatrix M_full repräsentiert. Die Metrikmatrix M_full weist somit k Zeilen und k Spalten auf und die Elemente der Metrikmatrix M_full sind Werte aus dem Zahlenraum R der reellen Zahlen. k entspricht dabei Anzahl der Trajektorien aus dem Trainingsdatensatz D.Subsequently, in a second method step V2, a metric value is successively formed for each pair of trajectories present in the training data set D and consisting of two trajectories T _j of the training data set D, according to which the trajectory set S is to be optimized. The metric value formed indicates the extent to which a trajectory prediction corresponds to ground truth. The metric value formed in each case is entered into a metric matrix M _full , with each element of the metric matrix M _full being assigned to the respective pair of trajectories. This therefore applies to the metric matrix M _full $$M_{full}\in R^{kxk}$$

where R represents the number space of real numbers and kxk represents the dimension of the metric matrix M _full . The metric matrix M _full therefore has k rows and k columns and the elements of the metric matrix M _full are values from the number space R of real numbers. k corresponds to the number of trajectories from the training data set D.

The metric according to which the trajectory set S is to be optimized is therefore calculated in pairs. This is carried out for all trajectories T _j in the training data set D, where the metric matrix M _full describes the pairwise metrics of two trajectories T _j . For example, for M _full [i, j], the metric value is between a trajectory i and a trajectory j. Thus, such a metric is always a measure between two trajectories T _j and is used in prediction to evaluate how good a prediction is compared to the ground truth. The metric can be, for example, an average displacement error (ADE for short), a final displacement error (FDE for short) or a miss rate. In addition, weighted combinations of such metrics can also be used if necessary. All metrics listed are permutation invariant, i.e. metric (i, j) = metric (j, i). If this is the case, a symmetric matrix M _full results.

wobei R wiederum den Zahlenraum der reellen Zahlen repräsentiert und k die Anzahl der Elemente des Metrikspeichers M_set repräsentiert. Der Metrikspeicher M_set enthält also k Elemente, die Werte aus dem Zahlenraum R der reellen Zahlen annehmen können. k entspricht wiederum der Anzahl der Trajektorien aus dem Trainingsdatensatz D. inf repräsentiert die Werte, mit denen die Elemente des Metrikspeichers M_set initialisiert werden. Die Bezeichnung „inf“ wurde in Anlehnung an den Begriff „infinite“ gewählt, um aufzuzeigen, dass die Initialisierung mit unendlich oder unrelalistisch hohen Werten erfolgen soll.In a third method step V3, a metric memory M _set of a current trajectory set S is created and initialized with arbitrarily high values, for example infinite or unrealistic high values. So M _set applies to the metric memory $$M_{set}=\left[inf,inf\dots \right]\in R^k$$

where R in turn represents the number space of real numbers and k represents the number of elements of the metric memory M _set . The metric memory M _set therefore contains k elements that can take values from the number space R of real numbers. k in turn corresponds to the number of trajectories from the training data set D. inf represents the values with which the elements of the metric memory M _set are initialized. The name “inf” was chosen based on the term “infinite” to show that the initialization should take place with infinitely or unrealistically high values.

Subsequently, in a fourth method step V4, a check is made in a branch as to whether at least one termination criterion is met. A number of trajectories T _j in the current trajectory set S and/or a last average error reached can be used as the termination criterion. If the at least one termination criterion is present, represented by a yes branch J, an existing, i.e. current, trajectory set S is selected as the metric-optimal trajectory set S. If the at least one termination criterion is not present, represented by a no branch N, a fifth method step V5 is repeated until at least one termination criterion is met.

In the fifth method step V5, a next optimal trajectory T _j is determined from the training data set D. This is done in one process Step V5.1 created and initialized a temporary memory list M _memory . The temporary memory list M _memory can always store three tuples comprising an iteration index j of the current trajectory T _j , a minimum M _{tmp, min,j} of the current trajectory T _j and an average value m _tmp , _min,j of the current trajectory T _j .

wobei R wiederum den Zahlenraum der reellen Zahlen repräsentiert und k die Anzahl der Elemente des Minimum M_tmp, _{min, j} repräsentiert, wobei k der Anzahl der Trajektorien aus dem Trainingsdatensatz D entspricht. Das heißt das Minimum M_tmp, _min,j ist eine vektorielle Größe mit k Elementen, die jeweils eine reelle Zahl sind.For the minimum M _tmp , _min,j holds $$M_{tmp,min,j}\in R^k$$

where R in turn represents the number space of real numbers and k represents the number of elements of the minimum M _tmp , _{min, j} , where k corresponds to the number of trajectories from the training data set D. This means that the minimum M _tmp , _min,j is a vector quantity with k elements, each of which is a real number.

wobei R den Zahlenraum der reellen Zahlen repräsentiert, Das heißt, der Mittelwert ist eine skalare Größe, deren Wert eine reelle Zahl ist.The mean m _tmp,min,j corresponds to the mean over all elements of the minimum M _tmp , _{min, j} . For the mean applies $$m_{tmp,min,j}\in R$$

where R represents the number space of real numbers, that is, the mean is a scalar quantity whose value is a real number.

All trajectories T _j in the training data set D are then iterated over, with an index j being assigned to each trajectory T _j . This iteration can also be parallelized on multiple CPU cores.

wobei j den Index der extrahierten Zeile repräsentiert, R den Zahlenraum der reellen Zahlen repräsentiert und k die Anzahl der Elemente des Metrikspeichers M_tmp,j repräsentiert und wobei k der Anzahl der Trajektorien aus dem Trainingsdatensatz D entspricht.In the iteration, a branch checks whether the index j is smaller than a count variable k. The counting variable k corresponds to the number of trajectories from the training data set D. If the index j is smaller than a counting variable k, a line M _full [j] with the index j from the metric matrix M _full is first created in a method step V5.2 extracted. The extracted line M _full [j] corresponds to the paired metric values of a current trajectory T _j to all other trajectories T _j in the training data set D. The extracted line M _full [j] is stored in a temporary metric memory M _tmp,j . The following applies to the temporary metric storage M _tmp,j : $$M_{tmp,j}=M_{full}\left[j\right]\in R^k$$

where j represents the index of the extracted row, R represents the number space of real numbers and k represents the number of elements of the metric memory M _tmp,j and where k corresponds to the number of trajectories from the training data set D.

The result corresponds to the paired metric values of a current trajectory T _j to all other trajectories T _j in the training data set D and is stored in a temporary metric memory M _tmp,j .

Subsequently, in a further method step V5.3, the minimum M _tmp,min,j of all entries in the metric memory M _set and in the temporary metric memory M _tmp,j is determined individually.

Subsequently, in a further method step V5.4, the mean value m _tmp , _min,j of all determined minima M _tmp , _min,j is formed, which results in an average error of the current trajectory set S in combination with a trajectory T _j of the current iteration. In theory, a set-based prediction model could achieve exactly this mean error, assuming that the prediction model always selects the best possible trajectory T _j for each scene.

The determined three-tuple is then selected in a further method step V5.5 from the iteration index j of the current trajectory T _j , the minimum M _tmp , _min,j of the current trajectory T _j and the mean value m _tmp , _min,j of the current trajectory T _j der added to temporary memory list M _memory .

Process steps V5.2 to V5.5 are repeated until the index j is greater than the counting variable k. If this is the case, in a further method step V5.6 the index j in the temporary memory list M _memory is determined, which has the lowest mean value m _tmp , _min,j .

Subsequently, in a further method step V5.7, the corresponding trajectory T _j with the index j is included in the trajectory set S and in a method step V5.8 the metric memory M _set is replaced by the determined minimum M _tmp , _min,j of the corresponding trajectory T _j .

If the training data set D is very large, for example with more than 200,000 trajectories T _j , then according to a possible embodiment of the present method, only a subset of the training data set D, also referred to as a subset, can be used to generate and select the metric-optimal trajectory set S. Under the condition that this subset also reflects the distribution of the entire training data set D well, a running time can be minimized without generating a significantly worse trajectory set S.

Trajectories T _j of vehicles are often noisy. Therefore, in a further possible embodiment of the present method it is provided that before using the training data set D to generate and select the metric-optimal trajectory set S, all trajectories T _j present in the training data set D are initially smoothed and a smoothed training data set D is created. This has the same number of trajectories T _j as the unfiltered training data set D. Kalman filters, for example with different variations such as extended Kalman filter, unscented Kalman filter and/or Kalman smoother, Savgol filter and other filters, can be used for this smoothing become.

When calculating the metric matrix M _full , the metric value of the smoothed trajectory T _j for all other trajectories in the training data set D is then determined in line j. The rest of the process is identical to the process described above.

After each iteration, the corresponding filtered trajectory Tj is then added to the trajectory set S. This ensures that the filtered trajectory T _j from the filtered training data set D is always iteratively added to the trajectory set S, which has an optimal effect on the original training data set D. Using smoothed trajectories T _j can provide greater generalization. Even if, for example, several noisy trajectories T _j have the same mean value, the exact courses of the trajectories T _j can be very different despite the same mean value due to the noise. If one selects an optimal trajectory T _j directly from these noisy trajectories T _j , high errors can arise from the surrounding trajectories T _j . However, if the trajectories T _j are first smoothed, a trajectory T _j can be found which approximately corresponds to the mean of all noisy trajectories T _j and thus covers them in a generalized manner.

The result after such smoothing shows 1 in the trajectory set S for vehicles or vehicle-like agent classes with, for example, 500 trajectories T _j .

Claims (4)

Method for selecting a trajectory set (S) from a predetermined training data set (D), characterized in that - in a first method step (V1) an initial and empty trajectory set (S) is created, - in a second method step (V2) one after the other for each in the training data set (D), each pair of trajectories consisting of two trajectories (T _j ) of the training data set (D), a metric value is formed, which indicates a measure to which extent a trajectory prediction corresponds to a ground truth, and the metric value formed in each case a metric matrix (M _full ) is entered, each element of the metric matrix (M _full ) being assigned to the respective pair of trajectories, - in a third method step (V3) a metric memory (M _set ) of the current trajectory set (S) is created and with high values is initialized, wherein a number of elements contained in the metric memory (M _set ) corresponds to a number of trajectories (T _j ) of the training data set (D), - in a fourth method step (V4) it is checked whether at least one termination criterion is met and - if the at least one termination criterion is present, an existing trajectory set (S) is selected as a metric-optimal trajectory set (S) and - if the at least one termination criterion is not present, a fifth method step (V5) is repeated until at least one termination criterion is met, and - in In the fifth method step (V5), a next optimal trajectory (T _j ) is determined from the training data set (D), whereby - a temporary memory list (M _memory ) is created and initialized, - across all trajectories (T _j ) in the training data set (D ) is iterated and for each trajectory (T _j ) a minimum (M _{tmp, min,j} ) is determined in the metric memory (M _set ) and the metric matrix (M _full ), - an average (m _tmp , _min,j ) of all determined minima (M _{tmp, min, j} ) is formed, - an average error of a current trajectory set (S) is determined from the mean value (m _{tmp, min, j} ) in combination with a trajectory (T _j ) of a current iteration, - an entry is determined in the temporary memory list (M _memory ) which has the lowest mean value (m _{tmp, min, j} ), - the metric memory (M _set ) is divided by the determined minimum (M _{tmp, min, j} ) of the metric memory ( M _set ) and the metric matrix (M _full ) is replaced and - the trajectory (T _j ), to which the determined minimum (M _{tmp, min, j} ) is assigned, is included in the trajectory set (S). Procedure according to Claim 1 , characterized in that a number of trajectories (T _j ) in the trajectory set (S) and / or a mean error reached last are used as the termination criterion. Procedure according to Claim 1 or 2 , characterized in that if a size of the training data set (D) is exceeded, a subset of the training data set (D) is used to generate and select the metric-optimal trajectory set (S). Method according to one of the preceding claims, characterized in that before using the training data set (D) to generate and select the metric-optimal trajectory set (S), all trajectories (T _j ) present in the training data set (D) are smoothed.

Images