CN117413257A - Method and system for testing driver assistance system for vehicle - Google Patents

Abstract

A computer-implemented method (100) for testing a driver assistance system (2) of a vehicle (1) in a simulated environment (3), having the following working steps: providing (101) a scene model defining input parameters required for defining a scene and their respective relation to other parameters of the scene model and defining some of said input parameters and their possible parameter values that may be changed for adjusting said scene; selecting (110) a combination of parameter values characterizing each respective scene by means of a combination test algorithm based on the scene model, wherein only interactions with a predetermined maximum number of other input parameters are considered in selecting the combination for the input parameter of interest; generating (111) a simulation environment (3) based on the selected parameter value combination; operating (112) a driver assistance system (2) to be tested in a simulation environment (3) according to the respective scenario; and the driving behavior of at least one driver assistance system (2) to be tested in the simulation environment (3) is detected (113).

Description

Method and system for testing driver assistance system for vehicle

Technical Field

The invention relates to a computer-implemented method for testing a driver assistance system of a vehicle in a simulated environment, wherein a scene model is provided, which specifies input parameters required for defining a scene and their respective relation to other parameters of the scene model and which specifies certain input parameters and possible parameter values that can be changed for adjusting the scene. The invention also relates to a corresponding system.

Background

Driver Assistance Systems (ADAS) implemented in the further development of Autonomous Driving (AD) are increasingly popular in the fields of passenger cars and commercial vehicles. Driver assistance systems make an important contribution to improving active road safety and are used to improve driving comfort.

In addition to ABS (antilock brake system) and ESP (electronic stability program) systems, which are mainly used for driving safety, a number of driver assistance systems are provided in the field of passenger cars and commercial vehicles.

Driver assistance systems that have been used to promote active road safety include parking assistance systems, adaptive cruise control systems, also known as Active Cruise Control (ACC), which adaptively adjusts a desired speed selected by a driver based on distance from a preceding vehicle. Another example of such driver assistance systems are an ACC automatic start-stop system that automatically keeps the vehicle running in addition to an ACC in a congested or parked situation, a lane keeping or lane changing assistance system that automatically keeps the vehicle in the lane, and a pre-crash protection system that prepares or initiates a brake to consume the kinetic energy of the vehicle in case of a possible crash, for example, and perhaps enables further measures in case of an unavoidable crash.

These driver assistance systems improve traffic safety by alerting the driver in critical situations until active intervention is initiated to avoid or mitigate the accident (e.g., by activating an emergency braking function). In addition, traveling comfort is improved by functions such as automatic parking, automatic lane keeping, and automatic distance control.

Only when the assistance provided by the driver assistance system is performed in a safe, reliable and as comfortable way will the occupant recognize the safety and comfort advantages of the driver assistance system.

In addition, each driver assistance system, depending on its function, must provide maximum safety for the own vehicle in the traffic scenario, without jeopardizing other vehicles or other traffic participants.

The respective degree of automation of the vehicle is divided here into so-called automation classes 1 to 5 (see for example standard SAE J3016). More particularly, the present invention relates to vehicles having driver assistance systems with an automation level of 3 to 5, which is generally considered to be autonomous driving.

The challenges of testing such systems are manifold. In particular, a balance must be found between test expenditure and test coverage. The main task in testing the ADAS/AD function is to prove that the driver assistance system function is ensured in all conceivable cases, in particular also in critical driving situations. This critical driving situation is at risk, since no or incorrect response of the respective driver assistance system may lead to accidents.

Thus, testing of driver assistance systems requires consideration of many driving situations that may occur in different scenarios. The space of variation of a possible scene is typically defined by a number of dimensions (e.g., different road characteristics, behavior of other traffic participants, weather conditions, etc.). From an almost infinite multidimensional parameter space, it is of particular relevance to driver assistance system testing to extract the parameter configuration of critical scenes that may lead to abnormal or dangerous driving situations.

As shown in fig. 1, the probability of such critical scenes occurring is much lower than that of normal scenes.

The scientific publication assumes that the autonomous operation of a vehicle is statistically safer than a human-driven vehicle only when the corresponding driver assistance system completes 2.75 hundred million miles of accident-free travel time to authenticate the respective driver assistance system. This cannot be achieved by real test driving per se, especially in the context of a time frame in which the development cycle and quality standards required by the automotive industry have been set to be urgent. It is also unlikely that enough key scenes or driving situations from such scenes are contained for the reasons described above.

The prior art discloses using actual test travel data of a measured fleet to verify and verify driver assistance systems and extract scenes from the collected data. Verification and verification using the full factor test plan is also disclosed.

Disclosure of Invention

The object of the invention is to be able to test driver assistance systems, in particular for autonomous driving, in a scene covering the entire multidimensional parameter space. In particular, the scene under test should be representative for the entire multidimensional parameter space. This object is achieved by the teaching of the independent claims. Advantageous designs can be found in the dependent claims.

A first aspect of the invention relates to a computer-implemented method for testing a driver assistance system of a vehicle in a simulated environment, having the following working steps:

providing a scene model defining input parameters required for defining a scene and their respective relation to other parameters of the scene model and defining certain input parameters and possible parameter values thereof that can be changed for adjusting the scene;

selecting, based on the scene model, a combination of parameter values characterizing each respective scene by means of a combination test algorithm taking into account boundary conditions, wherein only interactions with a predetermined maximum number of other input parameters are considered in selecting the combination for the input parameter of interest;

generating a simulated environment based on the parameter value combinations;

operating a driver assistance system to be tested in a simulated environment; and is also provided with

The driving behavior of at least one driver assistance system to be tested in a simulated environment is acquired.

A second aspect of the invention relates to a system for testing a vehicle driver assistance system in a simulated environment, having:

memory means for providing a scene model defining input parameters required for defining a scene and their respective relation to other parameters of the scene model and defining certain input parameters and possible parameter values that can be changed for adjusting a scene;

means for selecting combinations of parameter values characterizing respective scenes by means of a combination test algorithm based on the scene models, wherein only interactions with a predetermined maximum number of other input parameters are considered in selecting a combination for the input parameter of interest;

means for generating a simulated environment based on the selected parameter value combinations;

means for operating the driver assistance system to be tested in a simulated environment according to the respective scenario; and

means for capturing driving behavior of at least one driver assistance system to be tested in a simulated environment.

The vehicle environment in the sense of the invention is preferably formed at least by objects associated with the guidance of the vehicle by means of the driver assistance system. In particular, the vehicle environment includes a scene and a dynamic element. The scene preferably includes all stationary elements.

The scene in the sense of the invention is preferably formed in time sequence from a number of scenes, in particular static. The scene here specifies, for example, a spatial arrangement of at least one other object relative to the main object, for example, a distribution of traffic participants. The scenario may include, inter alia, a driving situation in which the driver assistance system is at least partially steering a vehicle equipped with the vehicle assistance system (i.e. "host vehicle"), e.g. autonomously performing at least one vehicle function of the host vehicle.

A driving situation in the sense of the present invention preferably describes a situation that should be taken into account when selecting a suitable driver assistance system behavior pattern at a certain moment. The driving situation is therefore preferably subjective, since it represents the perspective of the host vehicle. It also preferably includes the relevant conditions, likelihoods and influencing factors of the action. It is further preferred that the driving situation is deduced from the scene by means of an information selection process based on transient (e.g. emission specific) and constant targets and values.

The driving behavior in the sense of the present invention is preferably the behavior of the driver assistance system in the vehicle environment, which consists of actions and reactions.

The device in the sense of the present invention can be embodied in hardware and/or in software and in particular has a processing device, in particular a microprocessor device (CPU), and/or one or more programs or program modules, which are preferably connected in data or signal fashion to a memory system and/or a bus system. The CPU may be designed to process instructions implemented in the form of programs stored in the memory system, collect input signals from the data bus, and/or send output signals to the data bus. The memory system may have one or more, in particular different, storage media, in particular optical, magnetic, solid-state and/or non-volatile media. The program may be provided such that it performs or is capable of carrying out the methods described herein, such that the CPU is capable of performing the described steps of such methods.

The invention is based on the idea to limit or restrict the parameter space available for the configuration of scene parameters to the most relevant areas.

To this end, on the one hand, a combination test method is used, in which an algorithm is employed to generate the relevant combination of input parameters. The combined test algorithm aims to produce as high convincing as possible with as few tests as possible. For details of the combination test, reference is made to the publication "utility combination test" (D.R.Kuhn et al, NIST ad hoc publication 800/142).

Furthermore, only interactions with a predetermined maximum number of other input parameters are considered for the respectively concerned input parameters. The inventors have determined that it is sufficient to consider a limited number of parameters in consideration of all interactions between these parameters to realistically reflect the simulation. It is preferable that only interactions of three parameters are considered at a time. But then all parameters are taken into account during the combinatorial testing based on these interactions. In particular, the inventors have determined that the interaction between these parameters need not be considered by all factors.

While empirical results indicate that six-fold interactions need to be considered at best to cause various errors in the driver assistance system to be tested. This takes into account the fact that: not all parameters are related to causing errors in the driver assistance system to be tested.

The process according to the invention is preferably carried out fully automatically.

In an advantageous design of the method, the scene model is based on an ontology that specifies parameters that can be used to characterize the scene and the relationships or interactions between the different parameters.

Thus, one can consider in advance in the scene model: these parameters are not interrelated and therefore do not interact with each other. This results in further simplification of the simulation, which may save computing power or computing time.

An ontology in the sense of the present invention is preferably a formal ordered representation of a set of parameters in a traffic scene and the relationships or interactions that exist between them.

In a further advantageous design of the invention, the predetermined maximum number of other input parameters considered for interaction depends on the respective input parameter concerned.

As indicated above, all possible combinations of parameters or parameter values (combined intensities) to some extent are generally considered in the combined test.

According to this advantageous design, provision is made that the consideration of the interactions should depend on the respective input parameters concerned. It is thus considered that different input parameters are in different strong and weak relations with other parameters of the scene model, i.e. whether there is a parameter interaction and how strong such interaction is.

For input parameters where interactions are less pronounced, fewer interactions need to be considered. The number of parameters considered in particular with respect to the interaction can depend on the defined criteria. For example, not all parameter combinations may contribute to driver assistance system errors in generating a scene. Such parameter combinations or interactions can then always be ignored. This can be achieved by what is claimed in an advantageous design taking into account the different degree of interaction of the respective input parameters (mixing intensity).

The number of interactions of interest preferably depends on the criticality of the respective input parameter of interest. Criticality refers herein to the correlation of a parameter with the risk of potential accident of the driver assistance system.

Thus, a further significant reduction of test cases is obtained by taking into account the so-called mixing intensity, i.e. the number of considered interactions that vary depending on the input parameter of interest.

In a further advantageous design of the method, a hard boundary condition and a soft boundary condition are considered when selecting the parameter value combination, wherein the hard boundary condition excludes parameter value combinations that are practically impossible, and wherein the soft boundary condition excludes parameter value combinations that are practically feasible but contradict the vehicle action in the simulated environment. By taking the boundary conditions into account, the number of scenes to be simulated can be further reduced. The boundary conditions may be taken into account in the combined test algorithm, in particular when selecting parameter value combinations.

In another advantageous design, the scene model has a format that can be processed by an algorithm.

Such a format may for example be OWL, a so-called Web ontology language.

In a further advantageous embodiment of the method, the criticality of the input parameter of interest or of the parameter value of the input parameter of interest is determined by the following working steps:

selecting a combination of parameter values representing the respective scenes by means of a combination test algorithm in accordance with the scene model and taking into account boundary conditions, wherein only a unique interaction with other input parameters is considered in the selection of the combination for the input parameter of interest;

changing the parameter value combinations by means of an evolution algorithm, wherein a cost function is considered when changing, which is advantageous in terms of interactions that have not yet been considered and parameter combinations that are expected to be relevant for the critical scenario;

generating a simulated environment based on the changed parameter value combinations;

operating a driver assistance system to be tested in a simulated environment;

collecting driving behaviors of at least one driver assistance system to be tested in a simulation environment;

determining a scene goodness according to predetermined criteria related to the driving condition occurring, in particular the criticality of the driving condition occurring;

comparing the scene goodness with a threshold;

if the goodness is above the threshold, increasing a criticality score of the input parameter of interest or a parameter value of the input parameter of interest; and

a criticality score is output for the input parameter of interest or for a parameter value of the input parameter of interest.

The information obtained in this way about the criticality of the input parameter of interest or of the parameter range of the input parameter allows to determine the respective number of interactions of interest with other parameters.

In a further advantageous design, the method further comprises the following working steps:

the value of at least one of the input parameters of the scene is changed based on the determined goodness until at least one abort condition is met.

From this, extremely critical parameter values or value ranges can be determined.

In a further advantageous embodiment of the method, the change is performed by means of an intersection and/or a mutation, wherein the parameter values of a part of the input parameters are replaced by parameter values of another scene or the parameter values of at least one input parameter are replaced by new values.

Because of the crossover or mutation, the entire parameter space can be searched very reliably to find possible parameter combinations that are highly critical.

In a further advantageous embodiment of the method, the scene parameters are selected from the following group, depending on the type of driver assistance system to be tested:

the speed of the traffic participant and in particular the initial speed; the direction of movement and in particular the trajectory of movement of the traffic participants; a lighting condition; weather; road conditions; a temperature; the number and location of stationary and/or moving objects; the state and appearance of stationary and/or moving objects; the speed and direction of movement and in particular the trajectory of movement of the moving object; status of the signal device and in particular the optical signal device; traffic signs; number of lanes; acceleration or braking deceleration of a traffic participant or object; the phenomenon of dirt and/or aging of the lane, the geographical orientation of the traffic situation.

Drawings

FIG. 1 shows a graph of scene occurrence probability as a function of scene criticality;

FIG. 2 illustrates a block diagram of an embodiment of a method for testing a vehicle driver assistance system in a simulated environment;

FIG. 3a shows a first example of a first critical simulation scenario;

FIG. 3b shows a second example for a simulated virtual scene with a second criticality;

FIG. 4 illustrates an embodiment of a system for testing a vehicle driver assistance system in a simulated environment; and

fig. 5 shows an embodiment of an apparatus for operating a driver assistance system to be tested in a simulated environment.

Detailed Description

Fig. 1 shows scene occurrence probabilities that vary with scene criticality. The occurrence probability refers to the probability that a scene appears in real road traffic.

As shown in fig. 1, many scenes in real road traffic have less criticality, which also corresponds to the general life experience of the car driver. Such scene areas are denoted a in fig. 1. While highly complex scenes appear relatively rare, the areas of which are indicated by B in fig. 1. It is this highly complex scenario B that is particularly important for studying the functional effectiveness of driver assistance systems.

In order to obtain a wide variety of highly complex scenes B that are sufficiently numerous and diverse during driver assistance system testing, many scenes should be traversed based on the illustrated distribution curve.

A method for testing a vehicle driver assistance system in a simulated environment, thereby covering as many critical scenarios as possible, will be described below with reference to fig. 2-3 b:

an ontology-based scene model is preferably provided in a first working step 101. The scene model preferably specifies the input parameters required to define a scene and their respective relationships to other parameters of the scene model. Furthermore, the scene model preferably specifies input parameters and their possible parameter values that can be changed to adjust a scene. The relation between scene model parameters and their respective interaction strengths is preferably set by ontology.

The ontology is preferably built by describing the system environment. This forms the scene creation basis and represents a knowledge base of all elements that can be used to construct the scene. The body consists of two groups of elements: dynamic elements (e.g., people, moving objects, etc.) and static objects (e.g., fixed elements, trees, landmarks, etc.). The ontology is based on concepts that limit relationships between individual elements, where formal grammar is employed to demonstrate structure and inheritance between concepts. Specifically, the working steps in the ontology construction may be as follows:

first, a static portion of the ontology is constructed. This involves static parts of the concept and its elements, including relationships. The static part consists of environment, road infrastructure and conditions.

A portion of the environment is made up of fixed elements (e.g., ground, house, etc.).

These conditions include weather and lighting conditions. The road infrastructure part includes road segments, fixtures around the road such as buildings, trees, road signs. In addition, the static portion may contain sub-concepts such as ruts. The road sign may be, for example, part of a road segment concept.

Another working step is the construction of the dynamic part.

In a further working step, all parameters of each concept and parameter domain are defined. For example, the road segment has parameters such as heading angle, radius, gradient, length, etc. The roadmap has parameters such as color, type, style, bump, width, etc.

In a further working step, a dynamic part is constructed. It has a moving object, its behavior pattern and its actions. The main concept is that of a host vehicle, other vehicles and other moving objects such as pedestrians or animals. The moving object in turn has sub-concepts that in turn define properties and behavior patterns.

Finally, in a further working step, the relevant parameters are defined for each concept and each parameter field of the dynamic part. These parameters include acceleration, initial velocity, initial position, lane change speed, etc.

In another work step for forming the ontology, parameters are defined for each concept and parameter domain of the dynamic part. They are for example acceleration, initial speed, initial position, lane change speed, etc.

In a second working step 102 of the method 100, a combination of parameter values representing the respective scenes is selected by means of a combination test algorithm in accordance with the scene model and taking into account boundary conditions. Preferably, only unique interactions with other input parameters are considered when selecting a combination for the hypothetical input parameters.

In a third working step 103, the parameter value combination is changed by means of an evolution algorithm, wherein the change takes into account a cost function, which is advantageous for interactions that have not yet been considered and parameter combinations for which critical scenarios are expected.

In a third working step, a simulation environment is generated based on the parameter value combination. In a fourth working step 104, the driver assistance system to be tested is operated in a simulation environment. In a fifth step 105, the driving behavior of at least one driver assistance system to be tested in a simulated environment is detected. In a sixth step 106, the scene criticality is determined in accordance with predetermined criteria concerning the driver assistance system, in particular the system error or the risk of a collision or potential collision. In a further seventh working step 107, a criticality score of the input parameter value or parameter values of interest is determined or increased according to the scene criticality. In an eighth working step 108, the scene criticality is compared to a threshold. In a further ninth working step 109, the parameter value combination is changed by means of an evolution algorithm, wherein the change takes into account a cost function, which is advantageous in terms of interactions that have not yet been considered and parameter combinations for which critical scenarios are expected. As evolution algorithms, for example, intersections or abrupt changes are considered, wherein the parameter values of a part of the parameters are replaced by parameter values of another scene or the parameter values of at least one parameter are replaced by new values.

The subroutine comprising the working steps 102 to 109 is preferably repeated until a defined coverage is achieved in relation to the parameter space of the input parameter of interest. Furthermore, it is preferable to repeat the subroutine for each input parameter.

Interactions that have not been covered are studied by changing parameter value combinations by evolution. The parameter space for the respective input parameter of interest can be evaluated by means of the method 100 in conjunction with the cost function, in particular it being possible to identify what criticality the input parameter or the parameter range of the input parameter has.

In a tenth working step 110 of the method 100, a combination of parameter values representing each respective scene is selected by means of a combination test algorithm and based on the scene model. In this case soft and hard boundary conditions are preferably considered. The hard boundary condition excludes practically impossible combinations of parameter values. Furthermore, parameter values which are practically feasible but which violate the vehicle action in the simulated environment are preferably excluded by boundary conditions.

Examples of hard and soft boundary conditions are from fig. 4. Therefore, the movement trace of the host vehicle 1 cannot pass through the worksite, for example. In addition, the vehicle 1 cannot be in a position where another vehicle simulating traffic is located in the simulation. The soft boundary condition in fig. 4 is, for example, that the host vehicle does not travel in a distinct direction when it intends to eventually reach the destination location from the start location. Thus in fig. 4 the soft boundary condition is that the host vehicle is eventually driven towards the destination location, rather than in reverse, which is also favored from a hard boundary condition point of view.

The term "criticality" will be explained in connection with fig. 3a and 3 b.

Fig. 3a shows a first scene 3, when a pedestrian 6 crosses the road. The motorcycle 4 approaches the pedestrian 6 on a lane facing the pedestrian 6. Other vehicles 5b, 5c, 5d are also parked beside the lane, because these vehicles, the motor cycle 4, cannot see or hardly see the pedestrian 6. In a second lane for opposite traffic, another vehicle 5a travels parallel to the pedestrian 6. The host vehicle 1 approaches from behind the other vehicle 5a, and longitudinal and lateral control of the host vehicle is accomplished by the driver assistance system 2. It is unlikely that the arrangement according to fig. 3a is possible as to whether the motorcycle driver 4 is visible to the host vehicle 1 or its driver assistance system 2.

The other vehicles 5a, 5b, 5c, 5d, pedestrians 6, as well as the motorcycle driver 4 and the lane form a simulated environment for the host vehicle 1 or the driver assistance system 2 for maneuvering the host vehicle 1.

The parameters of scene 3 are, for example, the number of other vehicles, their location, the number of other traffic participants, their location and speed, ground conditions, markings on the road, etc. In fig. 3a, the host vehicle 1 or its driver assistance system 2 is ready to overrun other vehicles 5a participating in traffic. The motorcycle 4 reduces its speed so that the host vehicle 1 can pass by the other vehicles 5a.

In fig. 3b, the input parameters (i.e. the speed of the motorcycle 4) are changed such that the motorcycle does not decrease in speed, but continues to run at a constant speed. It is highly likely that a collision will occur between the host vehicle 1 and the motorcycle 4 when the host vehicle 1 performs an overtaking process with respect to the other vehicle 5a, as shown in fig. 3 b.

Because of the changing input parameters (i.e. the speed of the motorcycle 4 or its speed profile), a scene with high criticality is created. Thus, the speed of the motorcycle 4 is extremely important in terms of criticality, and the speed of the motorcycle 4 or its speed profile assumed in fig. 3b is highly critical.

Furthermore, it is preferred to consider only the interaction of the input parameter of interest with a predetermined maximum number of other input parameters when selecting the parameter value combination. In the example shown in fig. 3, the influence of the parameter "motorcycle 4 speed" or "road conditions" or "weather" on the parameter "motorcycle 4 braking distance" will be considered, for example. Whereas for the parameter "motorcycle 4 brake distance" the parameter "main vehicle 1 speed" will not be considered.

The number of combinations to be tested can thus be significantly reduced compared to the all-factor test. For example, in the case of 21 parameters of only one scenario, the number of tests can be reduced from 1.7 trillion tests in the case of full factors to only 1028297 tests when considering interactions with other parameters, respectively, i.e. six times interactions under all parameters, even to 125 tests when considering only dual interactions, i.e. dual interactions of the parameter of interest and the respective other parameters. Depending on the desired coverage, the reduction in the number of interactions considered contributes significantly to the reduction in computational effort. The inventors have determined that in most cases it is sufficient to consider the triplet interactions, i.e. the triplet interactions of the parameter of interest with the respective other two parameters, to adequately reflect reality.

It is also preferably provided that the predetermined maximum number of other input parameters when taking into account the interaction is dependent on the respective input parameter concerned. This approach therefore has a mixed combined intensity, which is always chosen according to the input parameters of interest. In this case, parameters with high scene criticality are preferably tested with higher intensity, i.e. more interactions with other parameters, than parameters with low criticality. With respect to fig. 4, for example, the speed of the motorcycle 4 is of significantly higher criticality than the position of, for example, the other vehicles 5b, 5c, 5 d.

The relevance or criticality of these input parameters may be determined in this case, as described with respect to the subroutines.

Based on the selected parameter value combinations, a simulation environment 3 is generated in an eleventh working step 111, as shown in fig. 3a and 3b and as already described for them.

In a twelfth working step 112, the driver assistance system 2 to be tested is operated in a simulated environment according to the respective scenario. For example, the host vehicle 1 in fig. 3a and 3b with the driver assistance system 2 to be tested completes the overtaking process. The overtaking process is part of the scene. Finally, in a thirteenth working step 113, the driving behaviour of the driver assistance system 2 to be tested in the simulated environment 3 is acquired.

From the collected driving behavior of the driver assistance system 2 to be tested, it can be determined whether the driver assistance system reacts or acts appropriately with respect to aspects such as active safety and passive safety and passenger comfort.

Fig. 5 illustrates an embodiment of a system 20 for testing a driver assistance system of a vehicle in a simulated environment.

Such a system preferably has memory means 11 for providing a scene model defining the input parameters required for defining a certain scene and their respective relation to other parameters of the scene model and defining certain input parameters and their possible parameter values that may be changed for adjusting a certain scene.

Furthermore, such a system 10 preferably comprises means 12 for selecting combinations of parameter values characterizing each respective scene by means of a combination test algorithm based on the scene model, wherein only interactions with a predetermined maximum number of other parameters are considered in selecting the combinations for the input parameters of interest.

Furthermore, such a system 10 preferably has means 13 for generating the simulation environment 3 based on the selected parameter value combination. The device 13 for generating the simulated environment 3 preferably has means 13-1 for generating the virtual environment 3 of the vehicle 1 on the basis of the selected parameter value combination. The interface 13-2 is set up for simulation or emulation of the virtual environment to the driver assistance system 2. Such an interface 13-2 may be, for example, a screen when the driver assistance system 2 has an optical camera.

In the example shown in fig. 5, the sensor of the driver assistance system 2 is a radar sensor, which emits a signal S. The signal S is collected by an interface 13-2 designed as a radar antenna.

The means 13-1 for generating the simulated environment 3 calculates a response signal S' based on the detected signal S and the simulated environment 3, which in turn is output via the radar antenna 13-2 to the radar of the driver assistance system 2. In this way, the driver assistance system 2 is tested.

For this purpose, further devices 14 are provided for operating the driver assistance system 2 to be tested in the simulation environment 3 according to the respective scenario. Such means 13 may be an interface or other mechanism for giving instructions to the driver assistance system 2, which operate the vehicle based on the instructions.

Finally, the system 10 preferably has means 15 for detecting the driving behavior of the at least one driver assistance system 2 to be tested in the simulation environment 3. If the driver assistance system is operating in a real environment or in the vehicle 1 on a test bench, such a device 15 may be a speed sensor and a sensor for determining a lateral steering of the vehicle. The device 15 can also be designed as a data interface if the driver assistance system 2 is running in a virtual environment or only testing the software of the driver assistance system (hardware in loop, software in loop).

The aforementioned means are preferably constituted by data processing means. However, the device 14 for operating the driver assistance system 2 to be tested in the environment 3 of the vehicle 1 can also be formed by a test bench, in particular for the driver assistance system 2 or the vehicle 1. In this case, the device 15 for detecting the driving behavior of the at least one driver assistance system to be tested in the simulation environment 3 can always be formed in part by sensors.

It is noted that these embodiments are merely examples and should not in any way limit the scope, applicability, or configuration of the invention. Rather, the foregoing description directs a skilled person to implement at least one embodiment wherein various changes may be made in the function and arrangement of elements described without departing from the scope of protection as set forth in the claims and the equivalent combination of features thereof.

List of reference numerals

A. B scene area

1. Vehicle with a vehicle body having a vehicle body support

2. Driver assistance system

3. Scene(s)

4. Motorcycle

5a, 5b, 5c, 5d other vehicles

6. Pedestrian

11. Simulation device

12. Driver assistance system operation device

13. Driving behavior monitoring device

14. Running state determining device

15. Goodness determination device

16. Scene changing device

17. Suspension condition inspection device

18. Interface

Claims (12)

1. A computer-implemented method (100) for testing a driver assistance system (2) of a vehicle (1) in a simulated environment (3), having the following working steps:

providing (101) a scene model defining input parameters required for defining a scene and their respective relation to other parameters of the scene model and defining some of said input parameters and their possible parameter values that may be changed for adjusting said scene;

selecting (110) a combination of parameter values characterizing each respective scene by means of a combination test algorithm based on the scene model, wherein only interactions with a predetermined maximum number of other input parameters are considered in selecting the combination for the input parameter of interest;

generating (111) a simulation environment (3) based on the selected parameter value combination;

operating (112) a driver assistance system (2) to be tested in the simulation environment (3) according to the respective scenario; and

the driving behavior of the at least one driver assistance system (2) to be tested in the simulation environment (3) is recorded (113).

2. The method of claim 1, wherein the scene model is based on an ontology defining parameters capable of characterizing the scene and relationships or interactions between the different parameters.

3. A method according to claim 1 or 2, wherein the predetermined maximum number of other input parameters when considering the interaction is dependent on the respective input parameter of interest.

4. Method according to one of the preceding claims, wherein the predetermined maximum number of other input parameters for which interactions with the respective parameter of interest are considered depends on the respective parameter of interest, in particular the criticality of the input parameter of interest.

5. Method according to one of the preceding claims, wherein a hard boundary condition and a soft boundary condition are considered when selecting parameter value combinations, wherein the hard boundary condition excludes parameter value combinations that are practically impossible, and wherein the soft boundary condition is a parameter value combination that is practically possible but violated with vehicle actions in the simulated environment.

6. The method according to one of the preceding claims, wherein the scene model has a format that can be processed by the algorithm.

7. Method according to one of claims 4 to 6, wherein the criticality of the input parameter of interest or the criticality of the parameter value of the input parameter of interest is determined by a subroutine having the following working steps:

selecting (102) a combination of parameter values representing the respective scenes by means of a combination test algorithm on the basis of the scene model and taking into account boundary conditions, wherein only a unique interaction with other input parameters is considered for the input parameters of interest when selecting the combination;

generating (103) a simulated environment based on the parameter value combinations;

operating (104) a driver assistance system to be tested in the simulation environment;

collecting (105) driving behavior of the at least one driver assistance system to be tested in the simulated environment;

determining (106) a scene criticality based on predetermined criteria associated with the driver assistance system, in particular a system error or a collision or collision risk;

determining or increasing (107) a criticality score of the input parameter of interest or a criticality score of a parameter value of the input parameter of interest in dependence of the scene criticality;

comparing the scene criticality to a threshold (108); and

the parameter value combinations are changed (109) by means of an evolution algorithm, wherein the change takes into account a cost function, which is advantageous in terms of interactions that have not yet been taken into account and parameter value combinations for which critical scenarios are expected, wherein the subroutine is repeated until a defined coverage is reached in relation to the parameter space of the input parameter of interest.

8. Method according to claim 7, wherein the change is performed by means of crossover and/or mutation, wherein the parameter values of a part of the parameters are replaced by parameter values of another scene or the parameter values of at least one parameter are replaced by new values.

9. The method (100) according to one of the preceding claims, wherein the parameters of the scene (3) are selected from the following group depending on the type of driver assistance system (2) to be tested:

the speed of the traffic participant and in particular the initial speed; the direction of movement and in particular the trajectory of movement of the traffic participants; a lighting condition; weather; road conditions; a temperature; the number and location of stationary and/or moving objects; the state and appearance of stationary and/or moving objects; the speed and direction of movement and in particular the trajectory of movement of the moving object; status of the signal device and in particular the optical signal device; traffic signs; number of lanes; acceleration or braking deceleration of a traffic participant or object; dirt and/or aging phenomena of the lane; geographic orientation of traffic conditions.

10. A computer program comprising instructions which, when run by a computer, cause the computer to perform the steps of the method according to one of claims 1 to 9.

11. A computer readable medium having stored thereon a computer program according to claim 10.

12. A system (10) for testing a driver assistance system (2) of a vehicle (1) in a simulated environment (3), having:

memory means (11) for providing a scene model defining input parameters required for defining a scene and their respective relation to other parameters of the scene model and defining some of said input parameters and their possible parameter values that are changeable for adjusting said scene;

means (11) for selecting a combination of parameter values characterizing each respective scene by means of a combination test algorithm based on the scene model, wherein only interactions with a predetermined maximum number of other input parameters are considered in selecting the combination for the input parameter of interest;

means (12) for generating the simulated environment (3) based on the selected parameter value combination;

means (13) for operating the driver assistance system (2) to be tested in the simulation environment (3) as a function of the respective scenario; and

means (14) for detecting the driving behavior of the at least one driver assistance system (2) to be tested in the simulation environment (3).