EP4302197A1

EP4302197A1 - Method and system for producing scenario data for the testing of a driver assistance system of a vehicle

Info

Publication number: EP4302197A1
Application number: EP22715966.2A
Authority: EP
Inventors: Tobias DÜSER
Original assignee: AVL List GmbH
Current assignee: AVL List GmbH
Priority date: 2021-03-01
Filing date: 2022-02-28
Publication date: 2024-01-10
Also published as: KR20230148366A; CN117222988A; WO2022183227A1; JP2024507997A; AT524821A1; US20240311279A1

Abstract

The invention relates to a system (10) for producing scenario data for the testing of a driver assistance system of a vehicle and to a corresponding method, the system comprising: means (11) for simulating a virtual traffic situation (3), wherein at least one first road user (1) can be controlled by a first user (2) and wherein simulation data are produced during the simulating; a first user interface (12) for outputting a virtual environment of the at least one first road user (1) to the first user (2) on the basis of the virtual traffic situation (3); a second user interface (13) for capturing inputs from the first user (2) for controlling the at least one first road user (1) in a virtual environment of the first road user (1); means (14) for checking the produced simulation data for an occurrence scenarios; means (15) for extracting scenario data relating to the scenario; and a data store (16) for recording the scenario data for the testing of the driver assistance system.

Description

Method and system for generating scenario data for testing a driver assistance system of a vehicle

The invention relates to a computer-implemented method for generating scenario data for testing a driver assistance system of a vehicle. Furthermore, the invention relates to a corresponding system.

The spread of driver assistance systems (Advanced Driver Assistance Systems - ADAS), which enable autonomous driving (Autonomous Driving - AD) in a further development, is increasing both in the field of passenger cars and in commercial vehicles. Driver assistance systems make an important contribution to increasing active traffic safety and serve to increase driving comfort.

In addition to the systems used in particular for driving safety, such as ABS (anti-lock braking system) and ESP (electronic stability program), a large number of driver assistance systems are offered in the area of passenger cars and commercial vehicles.

Driver assistance systems that are already being used to increase active road safety include a parking assistant, an adaptive cruise control system, also known as adaptive cruise control (ACC), which adaptively regulates a desired speed selected by the driver based on the distance from the vehicle in front. Another example of such driver assistance systems are ACC stop-and-go systems, which in addition to the ACC causes the vehicle to continue driving automatically in traffic jams or when vehicles are stationary, lane keeping or lane assist systems, which automatically keep the vehicle in its lane and pre-crash systems which, in the event of a collision, prepare or initiate braking, for example, in order to take the kinetic energy out of the vehicle, and initiate other measures if a collision is unavoidable.

These driver assistance systems both increase safety in traffic by warning the driver in critical situations and initiate an independent one Intervention to avoid or reduce accidents, for example by activating an emergency braking function. In addition, driving comfort is increased by functions such as automatic parking, automatic lane keeping and automatic distance control.

The safety and comfort gain of a driver assistance system is only perceived positively by the vehicle occupants if the support from the driver assistance system is safe, reliable and - as far as possible - comfortable.

In addition, each driver assistance system, depending on its function, must manage scenarios that occur in traffic with maximum safety for the vehicle and without endangering other vehicles or other road users.

The respective degree of automation of vehicles is divided into so-called automation levels 1 to 5 (see, for example, the SAE J3016 standard). The present invention relates in particular to vehicles with driver assistance systems of automation level 3 to 5, which is generally considered to be autonomous driving.

The challenges for testing such systems are manifold. In particular, a balance must be found between the test effort and the test coverage. The main task when testing ADAS/AD functions is to demonstrate that the function of the driver assistance system is guaranteed in all conceivable situations, especially in critical driving situations. Such critical driving situations have a certain degree of danger, since no reaction or an incorrect reaction of the respective driver assistance system can lead to an accident.

The testing of driver assistance systems therefore requires a large number of driving situations, which can arise in different scenarios, to be taken into account. The range of possible scenarios is generally spanned by many dimensions (e.g. different road properties, behavior of other road users, weather conditions, etc.). From this almost infinite and multidimensional parameter space, it is particularly relevant for testing the driver assistance systems, such parameter constellations for to extract critical scenarios that can lead to unusual or dangerous driving situations.

As shown in FIG. 1, such critical scenarios have a far lower probability of occurrence than usual scenarios.

Scientific publications assume that operating a vehicle in autonomous ferry operation is only statistically safer than a human-controlled vehicle if 275 million miles of accident-free driving time have been completed with the corresponding driver assistance system in order to validate the corresponding driver assistance system. This cannot actually be achieved using real test drives, especially given that the development cycles and quality standards required in the automotive industry set a very tight time frame. It would also be unlikely that a sufficient number of critical scenarios or driving situations resulting from these scenarios would be included for the aforementioned reason.

It is known from the prior art to use real test driving data from a real fleet of test vehicles to validate and verify driver assistance systems and to extract scenarios from the recorded data. Furthermore, it is known to use full factorial test plans for validation and verification.

It is an object of the invention to be able to test driver assistance systems, in particular autonomous driving functions, in a large number of scenarios. In particular, it is an object of the invention to generate scenarios for testing driver assistance systems.

This object is solved by the teaching of the independent claims. Advantageous configurations can be found in the dependent claims.

A first aspect of the invention relates to a computer-implemented method for generating scenario data for testing a driver assistance system of a vehicle, having the following work steps:

Generation of simulation data through the following sub-steps: Simulation of a virtual traffic situation, which has a plurality of virtual road users, wherein at least a first road user of the plurality of road users can be controlled by a first user and wherein those road users who cannot be controlled by users are automated, in particular by artificial intelligence or logic-based, to be controlled,

Outputting, on the basis of the virtual traffic situation, a virtual environment of the at least one first road user to the first user via a first, in particular at least visual, user interface, and

Recording of inputs from the first user to control the at least one first road user in the virtual environment of the first road user via a second user interface, with the simulation of the virtual traffic situation the recorded inputs of the first user and the resulting interaction of the at least one first road user with be taken into account in its virtual environment,

Checking the generated simulation data for the occurrence of scenarios which arise from the interaction of the at least one first road user with the virtual environment, the occurrence of the scenarios being characterized by a predefined constellation of simulated measured variables which preferably correspond to elementary maneuvers;

extracting, when occurrence of a scenario is determined, scenario data related to the scenario; and

Recording the scenario data for testing the driver assistance system.

The extracted scenario data are preferably output. This is preferably done via a user interface or a data interface.

A user within the meaning of the invention is a natural person, i. H. a human.

A driver assistance system within the meaning of the invention is preferably set up to support a driver when driving or to at least partially guide a vehicle, in particular a driver assistance system of automation level 3 to 5, or more particularly an autonomous driving function. Traffic user within the meaning of the invention is preferably any object that participates in traffic. In particular, a road user is a person, an animal or a vehicle.

For the purposes of the invention, extraction preferably means delimiting or isolating.

In particular, scenarios are delimited or isolated from the scenario data. In this case, data areas in the scenario data are preferably selected.

Scenario data within the meaning of the invention are preferably characterized by the position and movement of road users and the position of static objects in relation to a scenario.

A scenario within the meaning of the invention is preferably formed from a chronological sequence of, in particular static, scenes. The scenes give example, the spatial arrangement of the at least one other object relative to the ego object, z. B. the constellation of road users. A scenario preferably considers dynamic and static content. A model for the systematic description of scenarios is preferably used here, more preferably the model of the PEGASUS project (https://www.pegasusprojekt.de) with the following six independent levels: 1st street (geometry,...) ; 2. Street furniture and rules (traffic signs,...); 3. Temporary changes and events (road construction,...); 4. Moving objects (traffic-related objects such as: vehicles, pedestrians,... that move relative to the vehicle to be tested); 5. Ambient conditions (light situation, road weather,...); 6. Digital information (V2X, digital data / map,...). A scenario can contain, in particular, a driving situation in which a driver assistance system at least partially controls the vehicle, which is called the ego vehicle and is equipped with the driver assistance system, e.g. B. performs at least one vehicle function of the ego vehicle autonomously.

A traffic situation within the meaning of the invention preferably describes the totality of all circumstances in traffic with road users in a defined spatial area and/or in a defined period of time or point in time. These circumstances are preferably taken into account by road users for the selection of suitable behavior patterns at a specific point in time. A traffic situation preferably includes all relevant conditions, possibilities and determinants of actions. A traffic situation can, but does not have to, be represented from the point of view of a road user or object.

The simulated measured variables within the meaning of the invention are preferably selected from the following group: speed, in particular an initial speed, of a road user; a direction of movement, in particular a trajectory, of a road user; lighting conditions; Weather; road surface; Temperature; number and position of static and/or dynamic objects; a speed and a direction of movement, in particular a trajectory, of the dynamic objects; condition of signaling installations, in particular light signaling installations; traffic signs; number of lanes; Acceleration or deceleration of road users or objects.

A predefined constellation of measured variables within the meaning of the invention is preferably a constellation of values of one or more measured variables, in particular over time.

For the purposes of the invention, label means preferably provided with a categorizing designation.

A danger of a scenario within the meaning of the invention is preferably the spatial or temporal proximity to a traffic situation without a possible accident-free outcome (under one's own efforts and taking into account the uncertainties mentioned). If an accident can no longer be avoided, the danger is at its maximum. Danger is preferably also referred to as criticality. If the driving behavior or driving ability of a driver assistance system is taken into account, the danger can characterize a probability of an accident and/or a calculated duration up to a collision time. A maximum level of danger is preferably given when the calculated duration is 0 seconds and/or the accident probability P=1. An increased probability of an accident can be triggered in particular by a driving manoeuvre, for example an evasive reaction or strong gradient changes when steering, braking, accelerating (eg a vehicle gives way due to a strong steering movement). An increased accident probability can, in particular, also with regard to the other road users (who are guided based on logic or AI) and in a critical driving situation have to leave their driving order or their actual trajectory (by a evasive maneuvers). An increased probability of an accident can also arise, in particular, from external factors which affect the first road user or the other road users, for example if a driver is blinded. A quality within the meaning of the invention preferably characterizes the simulated scenario. A grade is preferably understood to mean a quality or nature and/or a relevance of the simulated scenario in relation to the dangerousness of a driving situation for a specific driver assistance system.

Relevance within the meaning of the invention is preferably understood to mean the frequency with which a scenario occurs in road traffic. For example, a backlit scenario is more relevant than a scenario in which an airplane lands on the street. The relevance preferably also depends on the region for which the road traffic is relevant. For example, there are scenarios that are relevant in Germany but not in China.

An area surrounding the vehicle within the meaning of the invention is preferably formed at least by the road users and other objects relevant to the vehicle guidance by the driver assistance system. In particular, the surroundings of the vehicle include scenery and dynamic elements. The scenery preferably includes all stationary elements.

The invention is based on the approach of using real people to generate scenarios, but no test drives in real traffic are required.

According to the invention, at least one real driver moves a vehicle in a virtual environment or a virtual environment. The invention makes the generation of scenarios accessible to a crowdsourcing approach. One or more users can now navigate a road user of their choice through virtual traffic situations on a simulator. Due to the almost endless possibility of options when navigating the road user(s) and other random mechanisms when simulating the virtual traffic situation, an almost endless number of different scenarios can arise, just like in real traffic. The invention uses predefined criteria to determine whether known or new scenarios occur. For this purpose, the simulation process and in particular the simulation data generated by it are continuously analyzed and monitored. With regard to the crowdsourcing approach, people's natural play instinct can be exploited. In this way, the method according to the invention or even a corresponding system can be made available to users. These users can then drive around in the simulated traffic "for fun". Alternatively, the users could also be given tasks, for example that they should get from a location A to a location B as quickly as possible while observing traffic regulations, or that they have to collect certain objects. Furthermore, the user could be distracted when navigating through the simulated traffic, e.g. B. in which he has to make certain voice inputs or the like.

The physics in the simulation correspond to reality in order to generate scenario data that is as realistic as possible. This applies in particular to the physical properties of road users and those of the environment. It is not possible to drive through objects or the like. More preferably, multiple users navigate multiple road users in the simulated traffic.

In an advantageous embodiment of the method, the scenario data generated in this way are already labeled, in particular objects of the virtual traffic situation are labeled. Information about the properties of objects is available in the simulation, so that the information can be assigned to the objects.

This is a particular advantage over data from a real test operation, in which all objects have to be labeled. This labeling is generally very complex, since it can only be carried out by humans.

In a further advantageous embodiment of the method, the scenario data are described during extraction in such a way that they can be used to simulate scenarios, preferably using OpenSCENARIO® or are output as OSI data. This means that the scenario data can be used directly to simulate scenarios.

In a further advantageous embodiment of the method, the user is encouraged to be active by various actions in the simulated virtual traffic environment. Such an activity can be, for example, the simulated behavior of another road user. In particular, these other Road users behave in such a way that the user has to react. In a further advantageous embodiment, this also has the following work steps:

Determining a quality of the extracted scenario data as a function of a predefined criterion, the quality preferably being characterized by the dangerousness of an underlying scenario. The quality indicates the quality of an underlying scenario. The extracted scenario data are preferably output when the quality reaches a termination condition. The quality is also preferably output to the user via a first or second user interface, in particular a display. A termination condition can preferably be a calculated duration up to a collision time or a collision probability.

In a further advantageous embodiment of the invention, the quality is higher the more dangerous the scenario that has arisen is, in particular the shorter the calculated duration up to a collision time.

In a further advantageous refinement of the method, a reward, in particular a fictitious one, is credited to the first user as a function of the quality of a scenario which has occurred. This gives the user motivation to create critical scenarios.

In a further advantageous embodiment of the method, a traffic flow model, in particular PTV-Vissim® or Eclipse SUMO, in particular version 1.8.0, is used to simulate the virtual traffic situation. A particularly realistic traffic situation can be generated by using a traffic flow model.

The features and advantages mentioned above in relation to the first aspect of the invention also apply correspondingly to the other aspects of the invention and vice versa.

A second aspect of the invention relates to a computer-implemented method for testing a driver assistance system of a vehicle, having the following work steps: Providing scenario data which characterize a scenario in which the vehicle is located and which has a plurality of other road users, the scenario data being generated using a method for generating scenario data according to the first aspect of the invention;

Simulating a virtual environment of the vehicle based on the provided scenario data;

Outputting the virtual environment to the driver assistance system via an interface; and

Operating the driver assistance system in the virtual environment of the vehicle.

In an advantageous embodiment of the method for testing a driver assistance system, the driver assistance system is simulated. This means that only the software or the actual code of the driver assistance system is taken into account or implemented when simulating the virtual traffic situation, according to the “software-in-the-loop” concept. In this way, a driver assistance system can be tested in a pure simulation.

In a further advantageous embodiment of the method for testing a driver assistance system, when the driver assistance system is operated, data relating to the vehicle's surroundings are fed into the driver assistance system and/or the driver assistance system, in particular its sensors, are stimulated on the basis of the vehicle's surroundings. This allows the driver assistance system, in particular its software or the entire hardware, to be tested on a test bench. In particular, a hardware-in-the-loop method can be used for this.

A third aspect of the invention relates to a system for generating scenario data for testing a driver assistance system of a vehicle, having:

Means for simulating a virtual traffic situation, which has a plurality of virtual road users, wherein at least a first road user of the plurality of road users can be controlled by a first user and wherein those road users who cannot be controlled by users are automated, in particular by artificial intelligence or logic-based, with simulation data being generated when simulating; a first, in particular at least optical, user interface for outputting, on the basis of the virtual traffic situation, a virtual environment of at least one first road user to the first user; and a second user interface for acquiring inputs from the first user for controlling the at least one first road user in a virtual environment surrounding the first road user, the means for simulating being further set up to use the acquired inputs from the first user and the resulting interaction of the at least to consider a first road user with his virtual environment when simulating the virtual traffic situation;

Means for checking the generated simulation data for the occurrence of scenarios which arise from the interaction of the at least one first road user with the rest of the environment, the occurrence of the scenarios being characterized by a predefined constellation of simulated measured variables which preferably correspond to elementary maneuvers;

means for extracting, when an occurrence of a scenario is determined by the means for checking the generated simulation data, scenario data related to the scenario; and a data memory for recording the scenario data for testing the driver assistance system.

Means within the meaning of the invention can be hardware and/or software and in particular one, preferably with a memory and/or bus system data or signal connected, in particular digital, processing, in particular microprocessor units (CPU) and/or one or more Have programs or program modules. In this case, the CPU can be designed to process commands that are implemented as a program stored in a memory system, to detect input signals from a data bus and/or to emit output signals to a data bus. A storage system can have one or more, in particular different, storage media, in particular optical, magnetic, solid state and/or other non-volatile media. The program may be designed to embody or perform the methods described herein is capable of, so that the CPU can execute the steps of such methods and then in particular can generate scenarios.

A fourth aspect of the invention relates to a system for testing a driver assistance system of a vehicle, comprising: a data memory for providing scenario data which characterize a scenario in which the vehicle is located and which has a plurality of other road users, the scenario data using a method according to any one of claims 1 to 8;

means for simulating a virtual environment of the vehicle based on the scenario data; and an interface for outputting the virtual environment to the driver assistance system in such a way that the driver assistance system can be operated in the virtual environment of the vehicle on the basis of the simulated scenario.

Further aspects of the invention relate to a computer program which contains instructions which, when executed by a computer, cause it to carry out a method according to the first or second aspect of the invention.

Further features and advantages result from the following description of exemplary embodiments with reference to the figures. It shows at least partially schematically:

FIG. 1 shows a diagram of the probability of occurrence of scenarios as a function of their criticality;

FIG. 2 shows a block diagram of an exemplary embodiment of a method for generating scenarios;

FIG. 3a shows a first example of a simulated virtual traffic situation;

FIG. 3b shows a second example of a simulated virtual traffic situation;

FIG. 4 shows an exemplary embodiment of a system for generating scenario data for testing a driver assistance system of a vehicle; FIG. 5 shows a block diagram of an exemplary embodiment of a method for testing a driver assistance system of a vehicle;

FIG. 6 shows an example of a simulated scenario; and

FIG. 7 shows an exemplary embodiment of a system for testing a driver assistance system of a vehicle.

Figure 1 shows the probability of occurrence of scenarios depending on the criticality of scenarios. The probability of occurrence is the probability of scenarios occurring in real road traffic.

It is noticeable in FIG. 1 that the majority of scenarios are of comparatively low complexity and/or criticality, which also corresponds to the general life experience of a car driver. The range of these scenarios is labeled "A" in FIG. In contrast, scenarios of high complexity and/or criticality, the area of which is labeled “B” in FIG. 1, occur comparatively rarely. However, it is precisely those scenarios “B” with great complexity and/or criticality that are highly relevant for investigating the functionality of driver assistance systems.

In order to achieve a sufficient number and diversity of different scenarios with high complexity "B" during the test of a driver assistance system, a very large number of scenarios must be run through based on the distribution curve shown.

A method for generating a large number of different scenarios for testing driver assistance systems is described below with reference to FIGS. 2 to 3b.

In a first work step 101, simulation data are generated. The first work step 101 preferably has three sub-processes.

In the first of these processes 101 -1 , a virtual traffic situation 3 is simulated, which has a plurality of virtual road users 1 , 4 , 5 a , 5 b , 5 c , 5 d , 6 . In this virtual traffic situation 3, at least one first road user 1 of the plurality of road users 1, 4, 5a, 5b, 5c, 5d, 6 can preferably be controlled by a first user 2 (cf. FIG. 4) and those road users 4, 5a, 5b , 5c, 5d, 6, which cannot be controlled by users, are controlled automatically. An artificial intelligence, a logical model or a traffic flow model, in particular PTV-Vissim® or Eclipse SUMO, is preferably used here. A large number of road users can preferably be located in the simulated virtual traffic situation 3 and are controlled by users, ie people.

There are essentially two approaches to simulating the virtual traffic situation: The simulation is based on data obtained in a real test drive. In this case, the parameters of individual objects, e.g. B. their speed can be changed by road users or be taken over as they were recorded during the real test drive. In an alternative embodiment of the method, the traffic situation 3 is created purely on the basis of mathematical algorithms. Both approaches can preferably also be mixed.

Such a simulated traffic situation 3 is shown, for example, in FIG. 3a. In the traffic situation 3 shown in FIG. 3a, a pedestrian 6 crosses a street. A vehicle 1 , which is controlled by the first user, approaches the pedestrian 6 in the lane facing the pedestrian 6 . Next to the traffic lanes, other vehicles 5b, 5c, 5d are parked, through which the pedestrian 6 is not or only poorly visible to a driver of the vehicle 1 controlled by the user. Another vehicle 5a is driving in the second lane for oncoming traffic at the level of pedestrian 6. A motorcyclist 4 is approaching behind the other vehicle 5a and is preparing to overtake the other vehicle 5a. Whether this motorcyclist 4 is visible to the driver of the vehicle controlled by the first user cannot be deduced from FIG. 3a.

The other vehicles 5a, 5b, 5c, 5d, the pedestrian 6 and the motorcyclist 4 form a virtual environment of the vehicle 1 controlled by the first user 2 in the traffic situation 3.

A driving situation or another scenario arise, which is dangerous or less dangerous. If the first user 2 becomes the vehicle 1 For example, as indicated in FIG. 3a by the bar in front of the movement arrow of the vehicle 1, by braking to a standstill, the motorcyclist 4 can overtake the oncoming vehicle 5a on the other lane 5a without being disturbed.

FIG. 3b shows the same virtual traffic situation 3 as FIG. 3a, in which the vehicle controlled by the first user 1 is in the same initial scenario as in FIG. 3a. As indicated by the movement arrow, which emanates from the vehicle 1 controlled by the first user, the first user continues to control this vehicle 1 with undiminished speed.

From there, it is highly probable that a subsequent driving situation or scenario will develop in which the motorcyclist 4 collides with the vehicle 1 controlled by the first user 1 . This is also indicated in FIG. 3b. Such a driving situation or such a scenario would correspond to a very high degree of danger.

In a second process 101 -2 of the first work step 101 , the virtual traffic situation 3 is output to the first user 2 via a first user interface 12 .

Possible user interfaces are shown as examples in Fig. 4 and preferably include optical user interfaces, in particular screens, audio user interfaces, in particular speakers, and/or a user interface for stimulating the sense of balance of the first user 2.

In a third process 101 -3 of the first work step 101, inputs by the first user 2 for controlling the at least one road user in a virtual environment of the first road user 1 are recorded via a second user interface 13 .

The second user interfaces 13 are also shown in FIG. 4 . It is preferably the steering wheel, a gear shift, a handbrake, a brake pedal, a clutch and/or an accelerator pedal and other possible control instruments that are available to a driver in a vehicle. Depending on which type of road user 1, 4, 5a, 5b, 5c, 5d, 6 the user 2 controls, however, other input means can also be present as the user interface 13, for example a type of joystick.

As already explained, the first road user 1 in FIGS. 3a and 3b is the black vehicle. When simulating the traffic situation 3 shown in FIGS. 3a and 3b, the recorded inputs of the first user 2 and the resulting interaction of the vehicle 1, i. H. of the first road user, with its virtual environment.

The interaction in the traffic situations 3 shown in FIGS. 3a and 3b is, for example, how the first user 2 reacts to the initial scenario. Depending on the reaction of the first user 2 to this initial scenario, the other road users, in particular the other oncoming vehicle 5a and the motorcyclist 4 and the pedestrian 6, will also react. For example, it is to be expected that the motorcyclist 4 will brake when he notices that the vehicle 1 controlled by the first user 2 is not reducing its speed. These interactions in turn have an impact on the development of the virtual traffic situation 3.

The work step of generating 101 simulation data is therefore a continuous process which, as indicated in FIG. 2, runs constantly in a loop and generates simulation data in the process.

During the simulation, the objects that are part of the virtual traffic situation 3 are already marked with meta information. A separate label is therefore no longer required. This affects both static and dynamic objects. Later data, which can be obtained from the simulation data, includes the so-called ground truth information.

If the scenario data is used, for example, to test a driver assistance system, it can be reconstructed which objects the driver assistance system correctly detected and which it incorrectly detected. Such labels are, for example, tree, pedestrian, car, truck, etc.

More preferably, 3 actions are set in the driving situations, which encourage the first user to be active. For example, this could be one in the Driving situation 3 of FIGS. 3a and 3b can be a vehicle which follows the vehicle 1 controlled by the first user 2 and urges it to accelerate. A surprising movement trajectory of the pedestrian 6, for example when he starts to run, can also be such an action.

In a second work step 102 of the method 100, the simulation data generated are checked for the occurrence of scenarios which arise from the interaction of the at least one first road user 1, the black vehicle in FIGS. 3a and 3b, with the virtual environment. Here, already known scenarios, which have already occurred earlier or are predefined as templates, can be checked, as well as scenarios that have not yet been predefined.

Both types of scenarios are preferably defined by predefined constellations of simulated measurement variables, which can be determined from the virtual traffic situation 3 . These predefined constellations either form the templates for scenarios or correspond to elementary maneuvers from which the occurrence of a scenario can be inferred. This could be, for example, a strong braking deceleration of the vehicle 1 in FIGS. 3a and 3b, which is used as a trigger condition for the occurrence of a scenario that has not yet been predefined.

If an occurrence of a scenario is determined, scenario data relating to the scenario are extracted in a third work step 103 . In this context, extraction means in particular delimiting or isolating a data area in the simulation data that is related to the determined scenario. The scenario data are preferably described during extraction in such a way that they are suitable for simulating scenarios. These can preferably be used with OpenSCENARIO® or OpenDrive®. More preferably, these are output as OSI data or OSI streams.

In a fourth work step 104 of method 100, the scenario data for testing the driver assistance system are recorded. This data is then available for testing a driver assistance system. Such a test method 200 is described further below with reference to FIG. In a fifth work step 105, a quality of the resulting scenario is preferably determined as a function of a predefined criterion, with the quality preferably being characterized by the dangerousness of one of the scenarios. The quality is preferably all the higher, the more dangerous the resulting scenario is. A degree of danger is preferably determined by so-called time-to-X metrics, such as those described in the publication “Metrics for assessing the criticality of traffic situations and scenarios”; P. Junietz et al., 11 . Workshop Driver Assistance Systems and Automated Driving”, FAS 2017. In particular, the following criteria can be used here: duration up to a point in time at which a collision occurs (time to collision), time to kickdown, time to steer, time to react, distance of closest encounter , Time to Closest Encounter, Worst Time to Collision. More preferably, the level of danger is characterized by a probability of an accident.

More preferably, a reward, in particular a fictitious one, is credited to the first user 2 as a function of the quality of the scenario that has occurred.

4 shows a system 10 for generating scenarios for testing a driver assistance system of a vehicle.

This system 10 preferably has means 11 for simulating a virtual traffic situation 3, which has a plurality of virtual road users. In order to make road users 1 controllable by a first user 2 , the system also has at least one first user interface 12 and at least one second user interface 13 .

The at least one first user interface 12 is used to output a virtual environment of at least one first road user 1 to the first user 2 . The virtual environment of the at least one first road user 1 is determined on the basis of the simulated virtual traffic situation 3 . It is essentially a representation of the virtual traffic situation 3 in the initial scenario from the point of view of the first road user 1, which the first user 2 controls.

As shown in FIG. 4, these user interfaces 12 are visual user interfaces such as screens and audio interfaces such as Loudspeakers and possibly devices with which the sense of balance of the respective user 2 can be influenced.

The second user interface or user interfaces 13 are set up to record inputs from the respective user 2 . As shown in FIG. 4, these are preferably different operating elements. As already explained above, these can be dependent on the respective road user 1 controlled by the user 2 . If the road user 1 controlled by the first user 2 is a vehicle, the user interfaces 12, 13 are preferably arranged in the area of a so-called seat box 19, which forms a simulator together with the user interfaces 12, 13, as shown in FIG.

Furthermore, the system 10 preferably has means 14 for checking the generated simulation data for the occurrence of scenarios. Furthermore, the system 10 preferably has means 15 for extracting scenario data relating to the scenario and a data memory 16 for recording the scenario data. Further preferably, the system 10 preferably has means for determining a quality of the extracted scenario data as a function of a predefined criterion. Furthermore, the system 10 preferably has a further interface 18 which is preferably set up as a user interface in order to output the quality to the user 2 and/or as a data interface in order to output the scenario data for further processing. The means 11, 14, 15, 16, 17, 18 are preferably part of a data processing device, which is preferably formed by a computer.

FIG. 5 shows a flow chart of an exemplary embodiment of a method 200 for testing a driver assistance system 7 of a vehicle 8, as illustrated in FIG.

In this method 200, scenario data, which characterize a scenario in which the vehicle 8) is located and which preferably has a plurality of other road users 4', 5a', 5b', 5c', 5d', 6', in a first step 201 simulated. These scenario data are also preferably based on simulations, from which they were extracted according to the method 100 described above. A scenario is simulated in a second step 202 on the basis of the scenario data. The vehicle 8 with the driver assistance system 7 to be tested is located in this scenario. In addition, the scenario preferably has a plurality of other road users or objects.

In the example of a scenario shown in FIG. 6, these are parked vehicles 5b′, 5c′, 5d′, a pedestrian 6′, an oncoming vehicle 5a′ in the other lane and a motorcycle 5′ also in this lane by analogy to the driving situations 3 shown in Figs. 3a, 3b.

On the basis of the simulated scenario, in a third work step 203, a virtual environment of the vehicle 8 is generated with the driver assistance system 7 and output.

In a third work step 203 , the virtual environment is output to driver assistance system 7 via an interface 23 . Finally, driver assistance system 7 is operated in the virtual environment of vehicle 8 in a fourth work step 204 .

The driving behavior of driver assistance system 7 in the scenario or environment can then be analyzed and evaluated. The driver assistance system 7 can be optimized on the basis of such an analysis or evaluation.

In the scenario shown in FIG. 6, the driver assistance system 7 of a vehicle 8 has a radar system which detects the objects located in the area surrounding the vehicle 8, in particular the road users 4', 5a', 5b', 5c', 5d', 6' detected.

In the illustrated example scenario, driver assistance system 7 is integrated into passenger vehicle 8 . Likewise, the driver assistance system to be tested could also be integrated into the motorcycle 4'. For example, the motorcyclist could be warned at an early stage by sensors in the driver assistance system and therefore not pull out. The driver assistance system of the motorcycle 4' reacts and the black car can continue driving without a collision occurring. A system 20 for testing a driver assistance system 7, which is suitable for executing the method 200 described with reference to FIGS. 5 and 6, is shown in FIG. Such a system 20 has a data memory 21 for providing scenario data which characterize a scenario in which the vehicle 8 is located. Means 22 are set up to simulate a virtual environment of the vehicle on the basis of the scenario data. Furthermore, the means 22 are set up to also render this environment.

Finally, an interface 23 is set up to output the virtual environment of a driver assistance system 7 . Such an interface can be a screen if the driver assistance system 7 has an optical camera. In the example shown in FIG. 7, the sensor of the driver assistance system is a radar sensor, which emits a signal S. This signal S is detected by radar antennas 23 .

On the basis of the detected signal and the simulated surroundings, the means 22 for simulating calculate a response signal S′, which in turn is output to the radar of the driver assistance system. In this way, the function of the driver assistance system 7 can be tested. Depending on which components of a driver assistance system 7 are to be tested, the simulated virtual environment, as shown in FIG. 7, can be tested by emulating signals to the sensor of the driver assistance system 7 . Alternatively, however, a signal can also be generated that is fed directly into the data processing unit 7 of the driver assistance system, or else a signal S′ that is only processed by the software of the driver assistance system 7 .

The data memory 21 and the means for simulating 22 are preferably part of a data processing device.

It is pointed out that the exemplary embodiments are only examples that are not intended to restrict the scope, application and structure in any way. Rather, the above description gives the person skilled in the art a guideline for the implementation of at least one exemplary embodiment, with various changes, in particular with regard to the function and arrangement of the components described, being able to be made without departing from the scope of protection as can be seen from the Claims and these equivalent combinations of features results.

Claims

patent claims

1. Computer-implemented method (100) for generating scenario data for testing a driver assistance system (7) of a vehicle (8), having the following work steps:

generating (101) simulation data

Simulating (101-1) a virtual traffic situation (3) which has a plurality of virtual road users (1, 4, 5a, 5b, 5c, 5d, 6), wherein at least a first road user (1) of the plurality of road users ( 1, 4, 5a, 5b, 5c, 5d, 6) can be controlled by a first user (2) and wherein those road users (4, 5a, 5b, 5c, 5d, 6) that cannot be controlled by users are automated , in particular controlled by artificial intelligence or logic-based,

Outputting (101 -2), on the basis of the virtual traffic situation (3), a virtual environment of the at least one first road user (1) to the first user (2) via a first, in particular at least optical, user interface (12), and

Recording (101-3) of inputs from the first user (2) for controlling the at least one first road user (1) in the virtual environment of the first road user (1) via a second user interface (13), wherein the simulation of the virtual traffic situation ( 3) the recorded inputs of the first user (2) and the resulting interaction of the at least one first road user (1) with his virtual surroundings are taken into account;

Checking (102) the generated simulation data for the occurrence of scenarios, which arise from the interaction of the at least one first road user (1) with the virtual environment, the occurrence of the scenarios are characterized by a predefined constellation of simulated measured variables, which preferably correspond to elementary maneuvers;

extracting (103), when occurrence of a scenario is determined, scenario data related to the scenario; and

Recording (104) the scenario data for testing the driver assistance system (7).

2. The method (100) according to claim 1, wherein objects of the virtual traffic situation (3) are labeled.

3. The method (100) according to claim 1 or 2, wherein the scenario data are described during extraction in such a way that they can be used to simulate scenarios, preferably using OpenSCENARIO®, or are output as OSI data.

The method (100) according to claim 1 or 2, wherein the first user (2) is stimulated into activity by an action or actions in the simulated virtual traffic environment.

5. The method (100) according to any one of the preceding claims 1 to 4, further comprising the following steps:

Determining (105) a quality of the extracted scenario data as a function of a predefined criterion, the quality preferably being characterized by the dangerousness of an underlying scenario.

6. The method (100) according to claim 5, wherein the quality is higher the more dangerous the respective scenario that has arisen is, in particular the shorter a calculated duration up to a collision time is.

7. The method (100) according to any one of the preceding claims 1 to 6, wherein the first user (2), in particular depending on the quality of a scenario that has occurred, is credited with a reward, in particular a fictitious one.

8. The method (100) according to any one of the preceding claims 1 to 7, wherein a traffic flow model, in particular PTV Vissim®, is used to simulate the virtual traffic situation (3).

9. Computer-implemented method (200) for testing a driver assistance system (7) of a first vehicle (8), having the following work steps:

Providing (201) scenario data which characterize a scenario in which the first vehicle (8) is located and which has a plurality of other road users (4', 5a', 5b', 5c', 5d', 6'), wherein the scenario data are generated by a method (100) according to any one of claims 1 to 8;

Simulating (202) a virtual environment of the first vehicle (8) on the scenario data provided;

Outputting (203) the virtual environment to the driver assistance system (7) via an interface (23); and

Operating (204) the driver assistance system (7) in the virtual environment of the first vehicle (8).

10. The method according to claim 9, wherein the driver assistance system (7) is simulated.

11. The method according to claim 10, wherein when the driver assistance system (7) is operated, data relating to the area surrounding the first vehicle (8) is fed into the driver assistance system (7) and/or the driver assistance system (7), in particular its sensors. be stimulated based on the environment of the first vehicle (8).

12. A computer program comprising instructions which, when executed by a computer, cause it to carry out the steps of a method according to any one of claims 1 to 11.

13. Computer-readable medium on which a computer program according to claim 12 is stored.

14. System (10) for generating scenario data for testing a driver assistance system of a vehicle, having:

Means (11) for simulating a virtual traffic situation (3), which has a plurality of virtual road users, wherein at least one first road user (1) of the plurality of road users (1, 4, 5a, 5b, 5c, 5d, 6) of can be controlled by a first user (2) and wherein those road users (4, 5a, 5b, 5c, 5d, 6) that cannot be controlled by users are controlled automatically, in particular by artificial intelligence or logic-based, wherein simulation data are generated during simulating; a first, in particular at least optical, user interface (12) for outputting, on the basis of the virtual traffic situation (3), a virtual environment of the at least one first road user (1) to the first user (2); and a second user interface (13) for detecting inputs from the first user (2) for controlling the at least one first road user (1) in a virtual environment of the first road user (1), wherein the means (11) are also set up for simulating in order to take into account the recorded input of the first user (2) and the resulting interaction of the at least one first road user (1) with his virtual environment when simulating the virtual traffic situation (3);

Means (14) for checking the generated simulation data for the occurrence of scenarios, which arise from the interaction of the at least one first road user (1) with the rest of the environment, the occurrence of Scenarios are characterized by a predefined constellation of simulated measured variables, which preferably correspond to elementary maneuvers;

means (15) for extracting, when an occurrence of a scenario is determined by the means (14) for examining the generated simulation data, scenario data related to the scenario; and a data memory (16) for recording the scenario data for testing the driver assistance system.

15. System (20) for testing a driver assistance system (7) of a first vehicle (8), comprising: a data memory (21) for providing scenario data which characterize a scenario in which the first vehicle (8) is located and which has a plurality of other road users (4', 5a', 5b', 5c', 5d', 6'), the scenario data being generated by means of a system (10) according to any one of claims 1 to 8;

Means (22) for simulating a virtual environment of the first vehicle (8) on the basis of the scenario data; and an interface (23) for outputting the virtual environment to the driver assistance system (7) in such a way that the driver assistance system (7) can be operated in the virtual environment of the first vehicle (8) on the basis of the simulated scenario.