DE102021133970A1 - Generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream - Google Patents

Abstract

The invention relates to a computer-implemented method for generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream, with the following method steps: simulated driving through at least part of a virtual simulation environment with a virtual vehicle (1) carrying a virtual sensor (2) by specifying a translatory Movement of the virtual vehicle (1) in the virtual simulation environment, wherein the virtual sensor (2) has a field of vision (3) in which it captures the virtual simulation environment, generating synthetic sensor data with the virtual sensor (2) by capturing the data from the virtual Vehicle (1) traveled through virtual simulation environment in the field of vision of the virtual sensor (2) and providing the synthetic sensor data as test data for testing a sensor data stream evaluating control system of a motor vehicle, wherein in the step of driving through at least part of the virtual simulation environment with the virtual sensor (2) supporting virtual vehicle (1), at least one vehicle dynamic movement caused by the translational movement of the virtual vehicle (1) is determined, the vehicle dynamic movement comprising a pitching and/or yawing and/or rolling and/or vertical movement of the virtual vehicle (1). , andin the step of generating the synthetic sensor data with the virtual sensor (2), the change in the visual range (3) of the virtual sensor (2) caused by the simulated vehicle dynamic movement of the virtual vehicle (1) is taken into account, in which the virtual sensor (1) the virtual simulation environment traveled through by the virtual vehicle (1) is recorded. In this way, a possibility is provided to obtain such test data for loading a control unit to be tested as part of a simulation, which is as close as possible to data that would have been obtained using a real system differ little.

Description

• simuliertes Durchfahren wenigstens eines Teils einer virtuellen Simulationsumgebung mit einem einen virtuellen Sensor tragenden virtuellen Fahrzeug durch Vorgabe einer translatorischen Bewegung des virtuellen Fahrzeugs in der virtuellen Simulationsumgebung, wobei der virtuelle Sensor einen Sichtbereich aufweist, in dem er die virtuelle Simulationsumgebung erfasst,
• Erzeugen synthetischer Sensordaten mit dem virtuellen Sensor durch Erfassen der von dem virtuellen Fahrzeug durchfahrenen virtuellen Simulationsumgebung in dem Sichtbereich des virtuellen Sensors und
• Bereitstellen der synthetischen Sensordaten als Testdaten zum Testen eines einen Sensordatenstrom auswertenden Steuersystems eines Kraftfahrzeugs.

The invention relates to a computer-implemented method for generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream, having the following method steps:

• simulated driving through at least part of a virtual simulation environment with a virtual vehicle carrying a virtual sensor by specifying a translational movement of the virtual vehicle in the virtual simulation environment, the virtual sensor having a field of vision in which it captures the virtual simulation environment,
• Generating synthetic sensor data with the virtual sensor by detecting the virtual simulation environment driven through by the virtual vehicle in the field of view of the virtual sensor and
• Providing the synthetic sensor data as test data for testing a control system of a motor vehicle that evaluates a sensor data stream.

The invention is relevant to the technical field of developing and testing driver assistance systems for autonomous driving or semi-autonomous driving of a motor vehicle. A motor vehicle that is suitable for autonomous or semi-autonomous driving generally has a sensor system for detecting objects in the area surrounding the vehicle and is at least partially controlled on the basis of signals detected by the sensor system. Corresponding sensors of such a sensor system of the motor vehicle are generally ultrasonic sensors, LIDAR sensors, RADAR sensors and/or cameras, with which the surroundings of the motor vehicle are recorded in order to achieve autonomous or at least partially autonomous driving of the test vehicle. Corresponding sensor systems are also systems for receiving data from other vehicles or stationary road traffic devices in order to achieve autonomous or at least partially autonomous driving of the vehicle by evaluating the data (Car2X communication).

With autonomous driving, the driver no longer has to intervene in the driving process, at least over long distances. In the semi-autonomous process, the vehicle at least reacts automatically, i.e. without intervention by the driver, to certain traffic situations and thus avoids a collision with a slower vehicle in front, for example, by automatically initiating a braking process when the vehicle in front is approaching. In order to be able to carry out such automated driving maneuvers, the sensors of the sensor system are used to detect where there are other vehicles, pedestrians or other obstacles in the vicinity of the vehicle.

Before a vehicle that is equipped with a corresponding sensor system can be operated in practice, i.e. on the real road network, for semi-autonomous or autonomous driving, a large number of tests must be carried out to ensure that the semi-autonomous or autonomous driving actually works is reliable and safe. Such tests, typically at an advanced stage, are also carried out on public roads with real vehicles. However, tests are usually carried out in advance in the form of simulations, with a virtual vehicle moving in a virtual three-dimensional simulation environment. In order to check the reaction of the sensor system for a large number of different, even unforeseen, situations, it is desirable to carry out such tests randomly and as realistically as possible.

It is therefore the object of the invention to provide a possibility, within the framework of a simulation, of obtaining test data for applying to a control unit to be tested which deviate as little as possible from data which would have been obtained using a real system.

This object is solved by the subject matter of the independent patent claims. Preferred developments can be found in the dependent claims.

• simuliertes Durchfahren wenigstens eines Teils einer virtuellen Simulationsumgebung mit einem einen virtuellen Sensor tragenden virtuellen Fahrzeug durch Vorgabe einer translatorischen Bewegung des virtuellen Fahrzeugs in der virtuellen Simulationsumgebung, wobei der virtuelle Sensor einen Sichtbereich aufweist, in dem er die virtuelle Simulationsumgebung erfasst,
• Erzeugen synthetischer Sensordaten mit dem virtuellen Sensor durch Erfassen der von dem virtuellen Fahrzeug durchfahrenen virtuellen Simulationsumgebung in dem Sichtbereich des virtuellen Sensors und
• Bereitstellen der synthetischen Sensordaten als Testdaten zum Testen eines einen Sensordatenstrom auswertenden Steuersystems eines Kraftfahrzeugs, wobei
• im Schritt des Durchfahrens wenigstens eines Teils der virtuellen Simulationsumgebung mit dem den virtuellen Sensor tragenden virtuellen Fahrzeug wenigstens eine durch die translatorische Bewegung des virtuellen Fahrzeugs hervorgerufene Fahrzeugdynamikbewegung ermittelt wird, wobei die Fahrzeugdynamikbewegung ein Nicken und/oder ein Gieren und/oder ein Wanken und/oder ein Vertikalbewegen des virtuellen Fahrzeugs umfasst, und
• im Schritt des Erzeugens der synthetischen Sensordaten mit dem virtuellen Sensor die durch die simulierte Fahrzeugdynamikbewegung des virtuellen Fahrzeugs verursachte Veränderung des Sichtbereichs des virtuellen Sensors berücksichtigt wird, in dem der virtuelle Sensor die von dem virtuellen Fahrzeug durchfahrene virtuelle Simulationsumgebung erfasst.

According to the invention, a computer-implemented method for generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream is provided, which has the following method steps:

• simulated driving through at least part of a virtual simulation environment with a virtual vehicle carrying a virtual sensor by specifying a translational movement of the virtual vehicle in the virtual simulation environment, the virtual sensor having a field of vision in which it captures the virtual simulation environment,
• Generating synthetic sensor data with the virtual sensor by detecting the virtual simulation environment driven through by the virtual vehicle in the field of view of the virtual sensor and
• Providing the synthetic sensor data as test data for testing a sensor data stream evaluating control system of a motor vehicle, wherein
• in the step of driving through at least part of the virtual simulation environment with the virtual vehicle carrying the virtual sensor, at least one vehicle dynamic movement caused by the translational movement of the virtual vehicle is determined, the vehicle dynamic movement being a pitching and/or yawing and/or rolling and/or comprises moving the virtual vehicle vertically, and
• in the step of generating the synthetic sensor data with the virtual sensor, the change in the visual range of the virtual sensor caused by the simulated vehicle dynamic movement of the virtual vehicle is taken into account, in which the virtual sensor detects the virtual simulation environment driven through by the virtual vehicle.

The present invention therefore also takes into account the dynamics of the vehicle and at least one movement associated therewith during the simulation. To do this, the forces acting on the vehicle must be determined. In order to specify this in a precisely defined manner, it is customary to use a coordinate system for the motor vehicle. With the introduction of such a vehicle coordinate system, a direction is defined for forces, moments, speeds and distances. This coordinate system is placed, for example, in the center of gravity of the vehicle, with the x-axis pointing in the direction of travel and the z-axis pointing upwards. The coordinate system is usually right-handed, so that the y-axis points to the left in the direction of travel. A movement in the x-direction is a longitudinal movement. If the vehicle moves in the y-direction, this is referred to as pushing. A movement in the z-direction is a stroke movement, also referred to below as vertical movement.

However, not only directed movements can occur on a moving vehicle, but also moments around the three coordinate axes mentioned above. In a motor vehicle, motion about the x-axis is called roll, motion about the y-axis is called pitch, and motion about the z-axis is yaw. There are six degrees of freedom in driving dynamics, namely three translatory (longitudinal movement, pushing, vertical movement) and three rotational (yawing, pitching and rolling). According to the invention, a translational movement of the virtual vehicle is specified on a virtual road in the x and y directions and the movements in at least one of the four other degrees of freedom are determined on the basis of the predefined translational movement of the virtual vehicle.

The determination of such vehicle dynamics movements on the basis of a predefined translatory movement is well known to the person skilled in the art from practice and therefore does not require any further explanation in the present case. For example, see the document "Christoph WEIDEMANN, State-of-the-Art Railway Vehicle Design with Multi-Body Simulation, Journal of Mechanical Systems for Transportation and Logistics, Vol. 3, No. 1, 2010”.

What is decisive for the invention is that in the step of generating synthetic sensor data with the virtual sensor, the change in the visual range of the virtual sensor caused by the simulated vehicle dynamic movement of the virtual vehicle is taken into account. This makes it possible to take the vehicle dynamics into account when generating the test data for testing the control system, which can be of particular interest in extreme situations, such as heavy braking, where the sensor tilts forward due to the vehicle pitching that the surroundings of the vehicle can only be detected in an area that is less far away from the vehicle than before, ie during normal travel of the vehicle.

In principle, the virtual sensor can be the simulation of different sensors carried by a motor vehicle. According to a preferred development of the invention, however, it is provided that the virtual sensor is a virtual ultrasonic sensor, a virtual LIDAR sensor, a virtual RADAR sensor or a virtual camera. Of course, the invention also includes situations in which a plurality of virtual sensors are carried by the virtual vehicle.

The virtual simulation environment can be designed in different ways. According to a preferred development of the invention, however, the virtual simulation environment is three-dimensional. It is particularly preferred that the three-dimensional virtual simulation environment includes a virtual road with a gradient on which the virtual vehicle is driving. In this way, not only can there be rolling movements, pitching movements and yaw movements, but also vertical movement of the vehicle.

According to a preferred development of the invention, when determining the transla vehicle dynamics movement caused by toric movement of the virtual vehicle takes into account: the mass of the virtual vehicle and/or the geometry of the virtual vehicle and/or the mass distribution of the virtual vehicle and/or the suspension of the virtual vehicle and/or the damping of the virtual vehicle. In particular, the more precisely the mass distribution of the virtual vehicle corresponds to the mass distribution of the real vehicle, the more realistically the movements of the vehicle can be simulated.

In principle, the virtual simulation environment could be created in real time as part of the simulation. According to a preferred development of the invention, however, it is provided that the virtual simulation environment is created in advance and stored on a non-volatile, computer-readable storage medium.

In principle, the attachment of the virtual sensor to the virtual vehicle can be flexible. According to a preferred development of the invention, however, it is provided that the virtual sensor is rigidly attached to the virtual vehicle. In this way, all movements of the virtual vehicle are transmitted directly to the virtual sensor.

The invention also relates to a non-transitory, computer-readable storage medium with synthetic sensor data stored thereon which have been obtained using a method as described above.

The invention also relates to using the synthetic sensor data stored on the non-volatile, computer-readable storage medium for testing a control system evaluating a sensor data stream. In such a use, the control system evaluating a sensor data stream is preferably a driver assistance system for a motor vehicle or a component of such a driver assistance system, for example a perception algorithm or a neural network set up as a classifier.

• 1 schematisch ein Verfahren gemäß einem bevorzugten Ausführungsbeispiel der Erfindung,
• 2a schematisch eine normale Fahrt eines virtuellen Kraftfahrzeugs und
• 2b schematisch eine Vollbremsung eines virtuellen Kraftfahrzeugs.

The invention is explained in more detail below using a preferred exemplary embodiment with reference to the drawings. In the drawings show:

• 1 schematically a method according to a preferred embodiment of the invention,
• 2a schematically a normal journey of a virtual motor vehicle and
• 2 B Schematic emergency braking of a virtual motor vehicle.

Out of 1 a method according to a preferred embodiment of the invention is shown schematically. This is a computer-implemented method for generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream.

In a first method step S 1 , at least part of a three-dimensional virtual simulation environment is simulated to be driven through with a virtual vehicle carrying a virtual sensor by specifying a translational movement of the virtual vehicle in the virtual simulation environment. The virtual simulation environment is created in advance and saved on a non-volatile, computer-readable storage medium, e.g. on the hard drive of a simulation computer.

The virtual vehicle drives on a virtual road with a gradient in the three-dimensional virtual simulation environment. The virtual sensor has a field of view in which it captures the virtual simulation environment. In this case, vehicle dynamic movements caused by the translational movement of the virtual vehicle are now determined, namely pitching, yawing, rolling and vertical movement of the virtual vehicle.

In the present case, the virtual sensor is a virtual LIDAR sensor that is rigidly connected to the virtual vehicle. In principle, however, the virtual sensor can also be a virtual ultrasonic sensor, a virtual RADAR sensor or a virtual camera. When determining the vehicle dynamic movement caused by the translatory movement of the virtual vehicle, the mass of the virtual vehicle, the geometry of the virtual vehicle, the mass distribution of the virtual vehicle, the suspension of the virtual vehicle and the damping of the virtual vehicle are taken into account.

In step S2, synthetic sensor data are generated with the virtual sensor by detecting the virtual simulation environment driven through by the virtual vehicle in the field of view of the virtual sensor. The change in the visual range of the virtual sensor caused by the simulated vehicle dynamics movements of the virtual vehicle is taken into account, in which the virtual sensor detects the virtual simulation environment driven through by the virtual vehicle. The field of vision of the virtual sensor is therefore not only given by the actual translational movement of the virtual vehicle within the simulation environment, but also by the vehicle dynamic movements. That's good For example, cornering or braking or accelerating affects the field of vision of the virtual sensor.

In step S3, the synthetic sensor data obtained are then provided as test data for testing a motor vehicle control system evaluating a sensor data stream, namely by storing them on a non-volatile, computer-readable storage medium, such as the hard drive of the simulation computer.

Finally, in step S4, the synthetic sensor data stored on the non-volatile, computer-readable storage medium is used to test a control system evaluating a sensor data stream or a component thereof. In the present case, the control system evaluating a sensor data stream is a driver assistance system for a motor vehicle, so that, as explained above, it can be tested under quasi-real conditions before it is actually used.

• 2a zeigt eine unbeschleunigte Fahrt eines virtuellen Fahrzeugs 1, also eine Fahrt mit konstanter Geschwindigkeit. Ein an dem virtuellen Fahrzeug 1 befestigter virtueller Sensor 2 hat einen vorbestimmten Sichtbereich 3 mit dem ein gewisser Bereich vor dem virtuellen Fahrzeug 1 erfasst wird. Kommt es nun zu einer starken Bremsung des virtuellen Fahrzeugs 1, wie bei einer in 2b dargestellten Vollbremsung, so verändert sich der Sichtbereich 3 des virtuellen Sensors 2. Durch das Nicken des virtuellen Fahrzeugs 1 neigt sich auch der virtuelle Sensor 2 nach vorne, so dass die Umgebung des virtuellen Fahrzeugs 1 nur noch in einem weniger weit von dem virtuellen Fahrzeug 1 entfernten Bereich als vorher während der unbeschleunigten Fahrt des virtuellen Fahrzeugs 1 erfasst wird. Der bei der Bremsung nicht mehr erfasste Bereich 4 ist in 2b schraffiert dargestellt. Für die Simulation der Fahrsicherheit kann die Berücksichtigung dieses veränderten Sichtbereichs 3 des virtuellen Sensors 2 sehr hilfreich sein.

In the step of generating the synthetic sensor data with the virtual sensor, the change in the visual range of the virtual sensor caused by the simulated vehicle dynamic movement of the virtual vehicle is taken into account, which is particularly helpful in extreme driving situations. In the case of heavy acceleration, heavy braking, but also rolling movements in curves, there can be a considerable change in the field of vision of the sensor attached to a vehicle. This is shown schematically in the 2a and 2 B shown:

• 2a shows an unaccelerated journey of a virtual vehicle 1, ie a journey at a constant speed. A virtual sensor 2 attached to the virtual vehicle 1 has a predetermined field of view 3 with which a certain area in front of the virtual vehicle 1 is detected. If the virtual vehicle 1 is now braked hard, as in the case of an in 2 B emergency braking shown, the field of view 3 of the virtual sensor 2 changes. Due to the nodding of the virtual vehicle 1, the virtual sensor 2 also inclines forward, so that the surroundings of the virtual vehicle 1 are only at a less distance from the virtual vehicle 1 distant area than before is detected during the unaccelerated running of the virtual vehicle 1 . Area 4, which is no longer covered during braking, is in 2 B shown hatched. Taking this changed field of view 3 of the virtual sensor 2 into account can be very helpful for the simulation of driving safety.

reference list

1

virtual vehicle

2

virtual sensor

3

Virtual sensor field of view

4

no longer covered area

Claims (11)

Computer-implemented method for generating test data for testing a control system of a motor vehicle that evaluates a sensor data stream, having the following method steps: Simulated driving through at least part of a virtual simulation environment with a virtual vehicle (1) carrying a virtual sensor (2) by specifying a translational movement of the virtual vehicle (1) in the virtual simulation environment, the virtual sensor (2) having a field of view (3) has, in which he captures the virtual simulation environment, Generating synthetic sensor data with the virtual sensor (2) by detecting the virtual simulation environment traveled through by the virtual vehicle (1) in the visual range of the virtual sensor (2) and Providing the synthetic sensor data as test data for testing a sensor data stream evaluating control system of a motor vehicle, wherein in the step of driving through at least part of the virtual simulation environment with the virtual vehicle (1) carrying the virtual sensor (2), at least one vehicle dynamic movement caused by the translatory movement of the virtual vehicle (1) is determined, the vehicle dynamic movement being a pitching and/or a yawing and/or a rolling and/or a vertical movement of the virtual vehicle (1), and in the step of generating the synthetic sensor data with the virtual sensor (2), the change in the visual range (3) of the virtual sensor (2) caused by the simulated vehicle dynamic movement of the virtual vehicle (1) is taken into account, in which the virtual sensor (1) the captured by the virtual vehicle (1) traveled through virtual simulation environment. Computer-implemented method claim 1 , wherein the virtual sensor (2) is a virtual ultrasonic sensor, a virtual LIDAR sensor, a virtual RADAR sensor or a virtual camera. Computer-implemented method claim 1 or 2 , where a plurality of virtual len sensors (2) is carried by the virtual vehicle. A computer-implemented method according to any one of the preceding claims, wherein the virtual simulation environment is three-dimensional. Computer-implemented method claim 4 , wherein the three-dimensional virtual simulation environment comprises a virtual road with a slope on which the virtual vehicle (1) travels. Computer-implemented method according to one of the preceding claims, wherein when determining the vehicle dynamics movement caused by the translational movement of the virtual vehicle (1), the following is taken into account: the mass of the virtual vehicle (1) and/or the geometry of the virtual vehicle (1) and/or the mass distribution of the virtual vehicle (1) and/or the suspension of the virtual vehicle (1) and/or the damping of the virtual vehicle (1). Computer-implemented method according to one of the preceding claims, wherein the virtual simulation environment is created in advance and stored on a non-volatile, computer-readable storage medium. A computer-implemented method according to any one of the preceding claims, wherein the virtual sensor (2) is rigidly attached to the virtual vehicle (3). A non-transitory, computer-readable storage medium having stored thereon synthetic sensor data obtained by the method of any preceding claim. Use of the on the non-volatile, computer-readable storage medium claim 9 stored synthetic sensor data for testing a sensor data stream evaluating control system. use after claim 10 , wherein the control system evaluating a sensor data stream is a driver assistance system for a motor vehicle.

Images