DE102021206588A1 - Procedure for trajectory optimization - Google Patents

Abstract

The invention relates to a method for determining a driving trajectory for recurring driving situations, the method having the following steps: (S10);- Detection of environmental information by a sensor system of the vehicle (F) while driving along the route (FS) (S11);- Storage of information on a traffic lane based on the environmental information (S12);- Determining a travel trajectory based on Information on the driving lane that can be driven on by means of a computing unit of the vehicle (F) which implements a strategy of reinforcement learning (S13); - storing the determined driving trajectory (S14).

Description

The invention relates to a method for trajectory optimization in recurring driving situations, in particular in recurring parking situations.

Partially automated or automated parking functions of vehicles are already known. For example, a trajectory traveled manually by the human driver from a starting position to a parking space is stored as the target position, including the parking maneuver and the detected environment. Based on this stored information, the stored trajectory can be traversed at a later point in time.

The disadvantage here is that the trajectory can only be followed as it was originally stored. Since the trajectory driven by the human driver is often not optimal in terms of length, time and/or the steering angle, and this manually driven trajectory forms the basis for the automated driving process, the automatically driven trajectory is often not optimal with regard to the aforementioned trajectory properties.

Proceeding from this, it is the object of the invention to specify a method for determining a driving trajectory for recurring driving situations, which offers improved trajectory properties for the driving route traveled automatically.

The object is solved by a method having the features of independent patent claim 1 . Preferred embodiments are the subject matter of the dependent claims. A system for determining a driving trajectory for recurring driving situations is the subject of the independent patent claim 15.

• Zunächst wird eine Fahrstrecke von einer Startposition zu einer Zielposition durch einen menschlichen Fahrer mittels eines Fahrzeugs abgefahren. Diese Fahrstrecke bezieht sich auf die wiederkehrende Fahrsituation, d.h. wird zukünftig öfter durchfahren, beispielsweise als Parkvorgang zu Hause, am Arbeitsplatz etc.

According to a first aspect, the invention relates to a method for determining a driving trajectory for recurring driving situations. The procedure has the following steps:

• First, a route is traveled from a starting position to a target position by a human driver using a vehicle. This route refers to the recurring driving situation, i.e. it will be driven through more often in the future, for example when parking at home, at work, etc.

While the route is being driven, information about the surroundings is recorded by a sensor system in the vehicle. The sensor system can be formed by any vehicle sensor system, via which the environment of the route can be detected and the collision-free driving lane can be determined therefrom. For example, the sensor system can include one or more ultrasonic sensors, at least one camera, at least one radar sensor and/or at least one LIDAR sensor.

After the environmental information has been recorded, information about a traffic lane that can be driven on is generated and stored based on this environmental information. This defines the area or driving path within which the trajectory can be planned.

A driving trajectory is determined by means of a computing unit of the vehicle based on the information on the driving lane that can be driven on. The computing unit implements a strategy of reinforcement learning, i.e. it searches for a driving trajectory that has improved trajectory properties.

Finally, the determined travel trajectory is stored in order to be able to drive the route automatically or partially automatically using a driver assistance system based on this travel trajectory in the future. The driver can be outside the vehicle, i.e. the vehicle drives itself to the parking position, for example.

The technical advantage of the method according to the invention is that the reinforcement learning method does not require complex training data to generate an optimized driving trajectory, but rather the human driver only has to travel the route at least once from a starting position to a target position in order to find the passable lane determine. An optimized tramline is then found by means of reinforcement learning. This has the advantage that the driving trajectory to be determined by the optimization is not limited by the quality of the training data, but a driving trajectory that is better than the driving trajectory driven by the human driver can be determined by the reinforcement learning.

According to one embodiment, an agent and an evaluation system are implemented in the computing unit. The agent is configured to determine a travel trajectory without training data, which is optimized with regard to predefined trajectory properties. A travel trajectory calculated by the agent is evaluated by the rating system based on trajectory properties ten assessed and depending on the assessment result, a new travel trajectory is calculated. As a result, new travel trajectories can be calculated, influenced by the assessment result, and travel trajectories with better trajectory properties can thereby be generated.

According to one exemplary embodiment, a number of different trajectory properties are used to assess a trajectory. The different trajectory properties can preferably be weighted differently. A new trajectory can be calculated under one or more specifications, in such a way that one or more trajectory properties are improved. This can be done, for example, in such a way that one or more trajectory properties are improved and other trajectory properties are degraded. For example, a very long trajectory can be improved by reducing the trajectory length in order to reach the target position more quickly.

According to one exemplary embodiment, the agent iteratively calculates new travel trajectories in such a way that the assessment result is increased. A number of different trajectory properties, for example with different weightings, are preferably included in the assessment. As a result, the trajectory can be improved over several iteration steps.

According to one exemplary embodiment, the evaluation system includes a reward function that calculates a positive or negative reward for a calculated travel trajectory. This reward is feedback information for the agent, which influences subsequent trajectory calculations. As a result, the agent can be influenced in such a way that successive trajectories with better trajectory properties and thus a better assessment result are calculated.

According to one embodiment, the agent iteratively calculates travel trajectories such that a subsequent travel trajectory receives a higher positive reward than a previous travel trajectory. The quality of the travel trajectories can thus be successively improved and a final travel trajectory which has sufficiently good trajectory properties can be determined by the reinforcing learning method.

According to one exemplary embodiment, the trajectory properties used to assess a driving trajectory include the time required to travel through the driving trajectory, the distance covered by the driving trajectory, information about steering angle changes, information about longitudinal acceleration and/or information about lateral acceleration. The travel trajectories can be assessed objectively by means of these trajectory properties, based on the perception of a human driver.

According to one exemplary embodiment, the route from the starting position to the target position is traversed several times. This can exclusively be a number of driving processes carried out by the human driver or driving processes that are also carried out at least partially in an automated manner. Information about the surroundings is recorded by a sensor system in the vehicle and information about the lane that can be driven on is determined and stored. More comprehensive environmental information can be obtained by driving along the route several times, so that an improved trajectory determination can take place.

According to one exemplary embodiment, a plurality of items of environmental information and/or information about the trafficable lane determined in chronological succession are combined, and this combined information about the trafficable lane is used to determine a travel trajectory. This fusion of environmental information or tramline information allows a modified, in particular larger, tramline to be obtained. Based on this modified tramline, there is more freedom for the calculation of travel trajectories.

According to one embodiment, when driving through the route from the starting point to the destination, the environment is continuously detected and a driving trajectory is determined by reinforcement learning if it can be seen from the detected environmental information that the information on the trafficable lane has changed due to static objects . It can thereby be ensured that the travel trajectory is continuously adapted to static changes in the driving lane that can be driven on, for example as a result of a structural change.

According to one exemplary embodiment, the route from the starting position to the target position is driven through automatically based on the determined travel trajectory. There is a deviation from the determined travel trajectory when an obstacle is detected on the determined travel trajectory. This makes it possible to react situation-specifically to temporary changes in the passable lane and avoid collisions by avoiding collisions.

According to one exemplary embodiment, the computing unit determines a number of different optimized driving trajectories using the reinforcement learning strategy, and a driving trajectory from these optimized driving trajectories is selected based on one or more target criteria. In this way it can be achieved that in cases in which a number of different optimized travel trajectories can be determined, that optimized travel trajectory which best satisfies predetermined assessment criteria is used.

• - Erfassen von Umgebungsinformationen durch eine Sensorik des Fahrzeugs während des Abfahrens der Fahrstrecke von einer Startposition zu einer Zielposition durch einen menschlichen Fahrer;
• - Abspeichern von Informationen zu einer befahrbaren Fahrgasse basierend auf den Umgebungsinformationen;
• - Ermitteln einer Fahrtrajektorie basierend auf Informationen zur befahrbaren Fahrgasse mittels der Recheneinheit, die eine Strategie des bestärkenden Lernens implementiert;
• - Abspeichern der ermittelten Fahrtrajektorie in einer Speichereinheit.

According to a further aspect, the invention relates to a system for determining a driving trajectory for recurring driving situations, comprising a computing unit provided in a vehicle. The procedure has the following steps:

• - Detection of environmental information by a sensor system of the vehicle while driving along the route from a starting position to a target position by a human driver;
• - Saving information on a traffic lane based on the environment information;
• - Determination of a travel trajectory based on information on the passable lane by means of the computing unit, which implements a strategy of reinforcement learning;
• - Saving the determined driving trajectory in a memory unit.

The terms “approximately”, “substantially” or “roughly” mean deviations from the exact value by +/-10%, preferably by +/-5% and/or deviations in the form of changes that are insignificant for the function .

Further developments, advantages and possible applications of the invention also result from the following description of exemplary embodiments and from the figures. All of the features described and/or illustrated are fundamentally the subject matter of the invention, either alone or in any combination, regardless of how they are summarized in the claims or how they relate back to them. The content of the claims is also made part of the description.

• 1 beispielhaft eine Draufsichtdarstellung einer Fahrstrecke einer wiederkehrenden Fahrsituation; und
• 2 beispielhaft ein Blockdiagramm, das die Verfahrensschritte zur Ermittlung einer Fahrtrajektorie für wiederkehrende Fahrsituationen verdeutlicht.

The invention is explained in more detail below with reference to the figures of exemplary embodiments. Show it:

• 1 by way of example, a plan view of a route of a recurring driving situation; and
• 2 a block diagram, for example, which illustrates the procedural steps for determining a driving trajectory for recurring driving situations.

1 shows an example and a rough schematic of a vehicle F at the beginning of a recurring driving situation, which is described by a route FS from a starting position SP to a target position ZP. Such a route FS can be, for example, the route between a property access road and a parking position provided on the property, for example a garage parking space.

In order to be able to drive through this route FS automatically, it is necessary for a computing unit of the vehicle F to calculate a travel trajectory, along which the vehicle F travels from the starting position SP to the target position ZP.

For the calculation of the driving trajectory, it is necessary that information about the environment in the area of the driving route FS is available in order to know the freely navigable area based on this, also referred to below as the driving path.

In order to determine the driving path, the vehicle F has a sensor system, by means of which the environmental information can be recorded. The sensor system can include, for example, ultrasonic sensors, one or more cameras, one or more radar sensors and/or one or more LIDAR sensors.

In order to be able to initially determine the environmental information along the route FS, it is necessary for the route FS to be traveled at least once by the human driver, i.e. from the starting position SP to the target position ZP. During this driving through, the environmental information can be recorded by the sensors of the vehicle F and the driving path that can be driven on can thus be determined.

After the travel path that can be driven on has been determined, a travel trajectory from the starting position SP to the target position ZP can be calculated. This is preferably done by a trajectory planner that is implemented in a computer unit of the vehicle.

The computer unit is configured for machine learning, based on the principle of reinforcement learning. In this case, the computer unit has an agent that can determine a travel trajectory without training data, which is optimized with regard to predefined trajectory properties. In other Words, the computer unit does not implement a supervised learning method based on a set of training data.

In addition, an evaluation system is provided, which is designed to evaluate a calculated travel trajectory based on predefined trajectory properties. Depending on how well the calculated driving trajectory fulfills the specified trajectory properties, a positive or negative reward is provided by the evaluation system. In particular, the rating system generates a positive reward if the trajectory properties of a newly calculated travel trajectory are better than the trajectory properties of a previously calculated trajectory. In the opposite case, the rating system generates a negative reward, for example, if the trajectory properties of a newly calculated travel trajectory are worse than the trajectory properties of a previously calculated trajectory. The previously calculated trajectory can in each case be the trajectory calculated directly beforehand, or a trajectory can be used that was calculated a long time ago.

Based on the positive or negative rewards, the agent independently learns how a driving trajectory has to be changed in order to receive positive rewards. The rewards can be used to derive a utility function that represents the value of a trajectory property in relation to receiving a positive reward. This makes it possible to achieve a targeted improvement in the trajectory properties.

The process for determining a travel trajectory is preferably initially initiated when a driving path that can be driven on is available for a repeated driving situation. For example, the calculation can be initiated immediately after reaching the target position ZP. Travel trajectories are preferably calculated until a sufficiently good travel trajectory can be determined. One or more termination criteria can be defined here, based on which the iterative trajectory optimization is terminated.

The trajectory properties on which the assessment of the travel trajectories is based can be, for example, the time to travel through the travel trajectory, the distance of the travel trajectory, information about the steering angle change, information about the longitudinal acceleration and/or information about the lateral acceleration.

The route FS is preferably traversed several times between the starting position SP and the target position ZP in order to

capture environmental information. As a result, different environmental information can be recorded in chronological succession. These can be combined or merged with each other in order to obtain improved information on the traffic lane that can be driven on. This improved information can then be used as a basis for calculating the driving trajectory. The advantage lies in the fact that with each additional trajectory driven, in particular manually driven trajectory, the driving path that can be driven on can be enlarged and thus there is also a higher potential for optimizing the driving trajectory through reinforcement learning.

A detection of the surroundings also takes place when driving along the route FS automatically on the basis of a previously calculated driving trajectory. On the one hand, this is done with the aim that obstructive objects located on the travel trajectory are detected and the vehicle can avoid them or stop in front of them. On the other hand, the environment is also detected during automated driving because new stationary objects are recognized on a driving path that was previously detected as passable, and thus a new trajectory optimization through reinforcement learning, taking into account the changed driving path, becomes possible. In other words, the calculation of the travel trajectory can be carried out again when the driving lane that can be driven on changes, in order to determine an optimized travel trajectory based on the changed environmental situation.

It can happen that the optimization of the travel trajectory by reinforcement learning does not lead to a global optimum of the travel trajectory, but only to a local optimum. Therefore, the trajectory optimization can preferably take place several times, for example based on other optimization criteria, in order to obtain several different optimized travel trajectories. These different optimized travel trajectories can then be compared with one another. In particular, the different optimized driving trajectories can be compared with each other based on predetermined criteria such as time to drive through the driving trajectory, distance of the driving trajectory, information about the steering angle change, information about the longitudinal acceleration and/or information about the lateral acceleration in order to determine a final driving trajectory, which is then used for the autonomous driving function is used. The autonomous driving function can in particular be "trained parking" in a recurring parking situation.

2 shows a diagram that explains the procedural steps for determining the travel trajectory.

First, a route is traveled from a start position to a target position by a human driver using a vehicle (S10).

While the route is being driven, information about the surroundings is recorded by a sensor system in the vehicle (S11).

Subsequently, information on a driving lane that can be driven on is stored based on the information surrounding the vehicle (S12).

A travel trajectory is then determined based on information on the traffic lane that can be driven on by means of a computing unit in the vehicle (S13). The computing unit implements a strategy of reinforcement learning, in which a calculated driving trajectory is assessed based on trajectory properties and iteratively optimized in such a way that the iteration steps try to successively improve trajectory properties of the driving trajectory.

Finally, the determined driving trajectory is stored (S14). As a result, the driving trajectory determined can be used for automated driving in the recurring driving situation.

The invention has been described above using exemplary embodiments. It goes without saying that numerous changes and modifications are possible without leaving the scope of protection defined by the patent claims.

Reference List

f

vehicle

FS

driving distance

SP

starting position

ZP

target position

Claims (13)

Method for determining a driving trajectory for recurring driving situations, the method having the following steps: - Departing a route (FS) from a starting position (SP) to a target position (ZP) by a human driver using a vehicle (F) (S10); - Detection of environmental information by a sensor system of the vehicle (F) while driving along the route (FS) (S11); - Storing information on a traffic lane based on the environmental information (S12); - Determining a travel trajectory based on information on the passable lane by means of a computing unit of the vehicle (F), which implements a strategy of reinforcement learning (S13); - Saving the determined driving trajectory (S14). procedure after claim 1 , characterized in that an agent and an evaluation system are implemented in the computing unit, that a travel trajectory calculated by the agent is assessed by the evaluation system based on trajectory properties and a new travel trajectory is calculated depending on the assessment result. procedure after claim 1 or 2 , characterized in that several different trajectory properties are used to assess a trajectory and a new travel trajectory is calculated under one or more specifications, in such a way that one or more trajectory properties are improved. procedure after claim 2 or 3 , characterized in that the agent iteratively calculates new travel trajectories in such a way that the assessment result is increased. Procedure according to one of claims 2 until 4 , characterized in that the evaluation system includes a reward function that calculates a positive or negative reward for a calculated travel trajectory. procedure after claim 5 , characterized in that a subsequent travel trajectory is selected by the agent on the basis of the amount of its reward. Method according to one of the preceding claims, characterized in that the trajectory properties used to assess a driving trajectory include the time required to travel through the driving trajectory, the distance covered by the driving trajectory, information on steering angle changes, information on longitudinal acceleration and/or information on transverse acceleration. Method according to one of the preceding claims, characterized in that the route (FS) from the starting position (SP) to the Target position (ZP) is traveled several times while environmental information is detected by a sensor system of the vehicle (F) and information about the passable lane is determined and stored. procedure after claim 8 , characterized in that a plurality of items of information relating to the trafficable lane which are determined one after the other are combined, and this combined information relating to the trafficable lane is used to determine a driving trajectory. Method according to one of the preceding claims, characterized in that when driving through the route from the starting point to the destination, the environment is continuously detected and the determination of a travel trajectory is performed again by reinforcement learning if it can be seen from the detected environmental information that the information has changed to the passable tramline by static objects. Method according to one of the preceding claims, characterized in that based on the determined travel trajectory, the route (FS) from the starting position (SP) to the destination position (ZP) is driven through automatically and the determined travel trajectory is deviated from if an obstacle on the determined travel trajectory is detected. Method according to one of the preceding claims, characterized in that the computing unit determines a number of different optimized driving trajectories using the reinforcement learning strategy and selects a driving trajectory from these optimized driving trajectories based on one or more target criteria. System for determining a driving trajectory for recurring driving situations, comprising a computing unit provided in a vehicle (F), the method having the following steps: - Detection of environmental information by a sensor system of the vehicle (F) while driving along the route (FS) from a starting position (SP) to a target position (ZP) by a human driver; - Saving information on a traffic lane based on the environment information; - Determination of a travel trajectory based on information on the passable lane by means of the computing unit, which implements a strategy of reinforcement learning; - Saving the determined driving trajectory in a memory unit.

Images