DE102022127857A1 - Method, system and computer program product for sensitivity analysis of application parameters for automated driving functions - Google Patents

Abstract

The invention relates to a method for sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) in at least one concrete scenario, in particular for testing and training the ADAS/ADS system, comprising:- generating data sets of application parameters;- creating test cases by means of a test strategy from a test agent, the test strategy selecting a concrete scenario and a data set of application parameters for each test case;- assigning one or more evaluation parameters to a test case;- carrying out a simulation by a simulation module with the created first test cases to determine simulation results;- carrying out an evaluation of the simulation results to determine evaluation results;- generating and outputting a sensitivity analysis of the selected data sets of the application parameters for the ADAS/ADS system from the output module, in particular in the form of diagrams and histograms for an application guide.

Description

The invention relates to a method, a system, an ADAS/ADS system and a computer program product for the sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS).

Modern vehicles are equipped with a variety of automated driving functions to support the driver while driving and increase safety. Driver assistance systems support, for example, speed and distance control as well as lane keeping and lane change functions. This leads to improved driving comfort and greater safety, especially when driving on the motorway and when overtaking.

However, this trend towards advanced driver assistance systems (ADAS) and automated driving systems (ADS) in motor vehicles, but also in aircraft or watercraft, requires extensive safety strategies, since the responsibility for driving the vehicle no longer lies entirely with the driver, but active functions are taken over by computer units in the vehicle.

In order to be able to adapt automated driving functions to the desired driving behavior of a vehicle and a specific vehicle type without changing the software code, ADAS/ADS systems contain application parameters. The application parameters are summarized in a data set that can be installed on the control unit with the driving function. The application parameters can be updated both for ADAS/ADS systems developed internally by an automobile manufacturer and for ADAS/ADS systems developed by suppliers. However, it is often not clear how the change in the application parameters affects the driving behavior in different traffic scenarios. In addition, as the number of application parameters increases, the mutual dependencies and interactions between the application parameters grow, which increases the complexity as well as the effort and the associated costs for the process of providing suitable application parameters for an ADAS/ADS system.

Virtual simulation methods are therefore increasingly being used for the application process, as simulation enables virtual test drives to be carried out efficiently and quickly. However, only a part of reality is ever reproduced by a simulation and, in particular, subjective impressions of human drivers regarding the driving behavior of an ADAS/ADS system can only be evaluated to a limited extent using a simulation. Therefore, the application parameters of an ADAS/ADS system are fine-tuned by test drives with prototype vehicles either on a test site or in road traffic. However, since the number of test cases that can actually be carried out is limited due to the high effort and associated costs, it is desirable to develop an efficient and cost-reducing procedure for fine-tuning the application parameters.

The EN 10 2019 126195 A1 discloses a method for the efficient, simulative application of automated driving functions in order to generate simulation processes in real time with reasonable computational effort. Already simulated parameter set scenario combinations are used.

The DE10 2021 200927 A1 discloses a method for analyzing a partially autonomous robot or vehicle, using a sensitivity analysis to show which factors dominate the system behavior and against which uncertainties the system is robust.

The DE 10308314 A1 discloses a statistical analysis of the variance of a technical output parameter depending on a technical input parameter taking into account a sensitivity of the technical output parameter.

The EN 10 2019 212830 A1 discloses a method for validating a neural network for a vehicle, wherein a trained neural network calculates a first trajectory which is compared with a comparison trajectory using a predetermined metric.

The object underlying the invention is to provide possibilities for the sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS), so that the application process for providing suitable application parameters requires less time and can be carried out with improved efficiency in order to increase the safety of ADAS/ADS systems and to save resources and costs.

This object is achieved with respect to a method by the features of patent claim 1, with regard to a system by the features of patent claim 10, with regard to an ADAS/ADS system by the features of patent claim 14, and with regard to a computer program product by the features of patent claim 15. The further claims relate to preferred embodiments of the invention.

The present invention can be used to carry out a sensitivity analysis with regard to the effects of changes in application parameters for an ADAS/ADS system. This makes the effects and effects on the driving behavior of an ADAS/ADS system due to changes in application parameters transparent and understandable. The virtual simulation allows the effects of changes in application parameters to be calculated quickly and efficiently. In particular, diagrams and histograms can be created as part of an application guide that reflect the effect of successively changing application parameters and thus represent an important decision-making aid for an application engineer. In a further development, it can also be provided that the method and system according to the invention is used to generate an optimal data set of application parameters for an ADAS/ADS system. Overall, resources can be saved because both the actual driving of test routes with standard traffic situations and with specific corner cases can be reduced.

• - Generieren von Datensätzen von Applikationsparametern;
• - Erstellen von Testfällen mittels einer Teststrategie von einem Testagenten, wobei die Teststrategie für jeden Testfall zumindest ein konkretes Szenario und einen Datensatz von Applikationsparametern auswählt;
• - Zuweisen von ein oder mehreren Bewertungsparametern zu einem Testfall;
• - Durchführen einer Simulation von einem Simulationsmodul mit den erstellten Testfällen zur Bestimmung von Simulationsergebnissen;
• - Durchführen einer Bewertung der Simulationsergebnisse durch ein Bewertungsmodul zur Bestimmung von Bewertungsergebnissen;
• - Generieren und Ausgeben einer Sensitivitätsanalyse der ausgewählten Datensätze der Applikationsparameter für das ADAS/ADS-System von dem Ausgabemodul, insbesondere in Form von Diagrammen und Histogrammen für einen Applikationsleitfaden.

According to a first aspect, the invention provides a method for sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) in at least one specific scenario, in particular for testing and training the ADAS/ADS system. A specific scenario represents a traffic event in a temporal sequence. The method comprises the following method steps:

• - Generating data sets of application parameters;
• - Creating test cases using a test strategy from a test agent, whereby the test strategy selects at least one concrete scenario and a data set of application parameters for each test case;
• - Assigning one or more evaluation parameters to a test case;
• - Performing a simulation of a simulation module with the created test cases to determine simulation results;
• - Performing an evaluation of the simulation results by an evaluation module to determine evaluation results;
• - Generating and outputting a sensitivity analysis of the selected data sets of the calibration parameters for the ADAS/ADS system from the output module, in particular in the form of diagrams and histograms for an application guide.

In a further development, it is planned that the test cases are stored in a test database, the application parameters in an application parameter database, the concrete scenarios in a scenario database, and the evaluation parameters in an evaluation database.

In an advantageous embodiment, it is provided that the test strategy and/or the test agent uses at least one software application with calculation methods and/or artificial intelligence algorithms to change the application parameters.

In a further embodiment, it is provided that the algorithms and calculation methods are designed as mean values, minimum and maximum values, lookup tables, models for expected values, linear regression methods, Gaussian processes, fast Fourier transformations, integral and differential calculus, Markov methods, probability methods such as Monte Carlo methods, temporal difference learning, extended Kalman filters, radial basis functions, data fields, convergent neural networks, deep neural networks, and/or feedback neural networks.

In particular, it is provided that the simulation module has exchangeable sub-modules, wherein a first sub-module is designed as an environment model module, a second sub-module as a driver model module and a third sub-module as a vehicle model module.

Advantageously, it is provided that the simulation module and/or the sub-modules are connected to sensors and/or a database in order to obtain further information for the creation of simulation models, wherein the simulation models are passed on to a driving function module for carrying out the simulation of a driver assistance function.

In a further embodiment, it is provided that the evaluation module is a driving function evaluation module for determining the performance and safety of a driving function using key performance indicators (KPIs) and/or a simulation evaluation module for determining the quality of the simulation. lation using simulation quality criteria (SQCs), and wherein the evaluation results include the key performance indicators (KPIs) and/or the simulation quality criteria (SQCs).

In particular, an application parameter represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius and/or a number.

Advantageously, the sensors are designed as radar systems, LIDAR systems for optical distance and speed measurement, image-recording 2D/3D cameras in the visible, IR and/or UV range, GPS systems, acceleration sensors, speed sensors, capacitive sensors, inductive sensors, voltage sensors, torque sensors, precipitation sensors and/or temperature sensors.

According to a second aspect, the invention provides a system for sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) in at least one specific scenario, in particular for testing and training the ADAS/ADS system. A specific scenario represents a traffic situation in a temporal sequence. The system comprises a test module with a test agent and a test strategy, a simulation module, an evaluation module and an output module. The test agent is designed to generate data sets of application parameters and to create test cases using a test strategy. The test strategy selects at least one specific scenario and a basic data set of application parameters for each test case. In addition, one or more evaluation parameters are assigned to a test case. The simulation module is designed to carry out a simulation with the created test cases to determine simulation results. The evaluation module is designed to carry out an evaluation of the simulation results to determine evaluation results. The output module is designed to generate and output a sensitivity analysis of the selected data sets of the application parameters for the ADAS/ADS system, in particular in the form of diagrams and histograms for an application guide.

In a further development, it is provided that the test cases are stored in a test database, the application parameters in an application parameter database, the concrete scenarios in a scenario database, and the evaluation parameters in an evaluation database.

In an advantageous embodiment, it is provided that the test strategy and/or the test agent uses at least one software application with calculation methods and/or artificial intelligence algorithms to change the application parameters.

In a further embodiment, it is provided that application parameters represent a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius and/or a number.

According to a third aspect, the invention provides an ADAS/ADS system for a motor vehicle, wherein the method according to the first aspect is used for testing and training the ADAS/ADS system.

According to a fourth aspect, the invention provides a computer program product comprising an executable program code configured to carry out the method according to the first aspect when executed.

The invention is explained in more detail below with reference to embodiments shown in the drawing.

• 1 ein Blockdiagramm zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 2 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte eines erfindungsgemäßen Verfahrens;
• 3 ein Blockdiagramm eines Computerprogrammprodukt gemäß einer Ausführungsform des dritten Aspekts der Erfindung.

It shows:

• 1 a block diagram for explaining an embodiment of a system according to the invention;
• 2 a flow chart explaining the individual method steps of a method according to the invention;
• 3 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

Additional features, aspects and advantages of the invention or embodiments thereof are explained in the following description in conjunction with the claims.

ADAS/ADS systems include application parameters that can be used to adapt the automated driving functions to the desired driving behavior of a vehicle and a specific vehicle type without changing the software code. The application parameters are summarized in a data set that can be installed on the control unit with the driving function. The application parameters can be updated both for ADAS/ADS systems developed internally by an automobile manufacturer and for ADAS/ADS systems developed by suppliers. However, it is often not clear how changing the application parameters affects driving behavior in different traffic scenarios. In addition, as the number of application parameters increases, the mutual dependencies and interactions between the application parameters increase, which increases the complexity, effort and associated costs of the process of providing suitable application parameters for an ADAS/ADS system.

According to the invention, a sensitivity analysis of application parameters of an ADAS/ADS system in various driving scenarios is therefore carried out by means of a virtual simulation in order to create the basis for fine-tuning the application parameters of the ADAS/ADS system.

1 shows a system 100 according to the invention for the sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS), which uses a modular simulation approach and can be adapted for a variety of designs of ADAS/ADS systems. The system 100 according to the invention comprises a test agent module 200 with a test agent 220 and a test strategy 230, an application parameter database 320, a scenario database 330, an evaluation database 340, a test database 300, a simulation module 400, an evaluation module 500 and an output module 700, each of which can be provided with a processor and/or a memory unit.

In the context of the invention, a "module" can be understood as, for example, a processor and/or a processor unit and/or a memory unit for storing program instructions. The processor is specifically designed to execute the program instructions in such a way as to implement or realize the method according to the invention or a step of the method according to the invention. In particular, a module can be integrated into a cloud computing infrastructure.

In the context of the invention, a “processor” can be understood as, for example, a machine or an electronic circuit. A processor can in particular be a main processor (Central Processing Unit, CPU), a microprocessor or a microcontroller, for example an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions. A processor can also be understood as a virtualized processor, a virtual machine or a soft CPU. It can also be, for example, a programmable processor that is equipped with configuration steps for carrying out the above-mentioned method according to the invention or is configured with configuration steps such that the programmable processor implements the inventive features of the method, the system, the modules, or other aspects and/or partial aspects of the invention. In particular, the processor can contain highly parallel computing units and powerful graphics modules.

In the context of the invention, a "memory unit" or a "memory module" can be understood to mean, for example, a volatile memory in the form of a random access memory (RAM) or a permanent memory such as a hard disk or a data carrier or a removable memory module. The memory module can also be a cloud-based storage solution.

The term “database” refers to both a storage algorithm and the hardware in the form of a storage unit. In particular, the databases 300, 320, 330, 340 can be designed as part of a cloud computing infrastructure.

In the context of the invention, data refers to both raw data and already processed data from measurement results from sensors and from other data sources.

The test agent 220 of the test agent module 200 creates a plurality of relevant test cases T for one or more driving tasks using the test strategy 230 according to the invention. These relevant test cases T form the basis for an automatically generated application guide that can be used by an application engineer, for example. The respective driving task is formulated before starting the simulation, for example by an expert such as the application engineer. However, it can also be provided that a list of driving tasks is generated by a software application. This list can then be processed successively by the system 100 according to the invention. An example driving task is a lane change on a highway or a cut-in scenario.

The test cases T are created on the basis of selected concrete scenarios SZc in which a sensitivity analysis of the application parameters Papp is to be carried out. These Specific scenarios SZc are selected, for example, by an application engineer. In the context of the invention, a scenario SZ refers to a traffic event in a temporal sequence. An example of a scenario SZ is driving over a motorway bridge, turning into a turning lane, driving through a tunnel, turning into a roundabout or stopping in front of a pedestrian crossing. In addition, specific visibility conditions, for example due to twilight or high levels of sunlight, as well as environmental conditions such as the weather and the time of year, the volume of traffic and certain geographical topographical conditions can influence a scenario. For example, an overtaking maneuver can be described as a scenario in which a first vehicle is initially behind another vehicle, then changes lanes onto the other carriageway and increases speed to overtake the other vehicle. Such a scenario is also referred to as a cut-in scenario.

In the context of the invention, a parameterized or logical scenario SZp refers to a scenario that is written in particular in a machine-readable code. A parameterized scenario SZp is defined by various parameters P ₁ , P ₂ , ..., P _n from a set of possible scenario parameters P and associated scenario parameter values PV ₁ , PV ₂ , ..., PV _n from a set of possible scenario parameter values PV, where scenario parameter values PV specify the value range of a scenario parameter P _i . In the context of the present invention, scenario parameters P thus characterize properties and characteristics of a scenario. An example of a scenario parameter P _i is the speed of an ego vehicle, and a value range of the associated parameter value PV _i can cover the range from 100 km/h to 180 km/h for a scenario SZp _i . For another scenario SZp _k, the value range of the parameter value “speed of the ego vehicle” can extend from 40 km/h to 70 km/h.

While in a parameterized scenario SZp the scenario parameters P and the associated scenario parameter values PV are not all specified, a concrete scenario SZc is a scenario SZ in which the concrete scenario parameters Pc and the associated concrete scenario parameter values PVc or concrete value ranges of the scenario parameter values PV are specified. Both scenarios, the parameterized scenario SZp and the concrete scenario SZc, are scenarios that are written in particular in a machine-readable code. An example of concrete scenarios is a cut-in scenario with different speeds of the ego vehicle and a target vehicle as well as different vehicle types.

Various sources such as requirement specifications, expert knowledge, and/or measurements in public traffic or on test sites using sensors can be used to create parameterized scenarios SZp and concrete scenarios SZc. The sensors used can be designed in particular as radar systems, LIDAR systems for optical distance and speed measurement, image-recording 2D/3D cameras in the visible, IR and/or UV range, and/or GPS systems. Furthermore, acceleration sensors, speed sensors, capacitive sensors, inductive sensors, voltage sensors, torque sensors, precipitation sensors and/or temperature sensors, etc. can be used. In this way, a suitable parameterized scenario SZp _i or concrete scenario SZc _i can be derived from recorded data at a specific geographical location using a software application. The parameterized scenarios SZp and/or concrete scenarios SZc are stored in the scenario database 330.

Within the scope of the present invention, the test agent 220 creates test cases T _i that are relevant for creating an application guide for an ADAS/ADS system. In particular, it is intended that the relevant test cases T _i cover the traffic situations defined in the "Operational Design Domain" (ODD) that are important for assessing the performance and functionality of one or more automated driving functions or the overall performance of an ADAS/ADS system.

Furthermore, 230 data sets of Papp application parameters are generated for the test strategy. The Papp application parameters are, for example, a physical quantity, a chemical quantity, a torque, a speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius and/or a number. The data sets of the Papp application parameters can, for example, be developed from a basic data set that was generated as part of a general simulation of the ADAS/ADS system.

The test strategy 230 uses in particular algorithms based on combinatorics to generate data sets with different application parameters Papp. For example, one or more applications tion parameter Papp of the basic data set can be increased or decreased by a defined amount. However, other algorithms and calculation methods such as mean values, minimum and maximum values, lookup tables, models for expected values, linear regression methods, Gaussian processes, fast Fourier transformations, integral and differential calculus, Markov methods, probability methods such as Monte Carlo methods, temporal difference learning, extended Kalman filters, radial basis functions, data fields, convergent neural networks, deep neural networks, and/or feedback neural networks can also be used. This results in a large number of different data sets.

The test agent 220 assigns one or more selected concrete scenarios SZc _i to the respective data sets, so that a set of test cases T is created. In addition, one or more evaluation parameters Pval are assigned to each test case T _i , so that each test case T _i consists of a data set of application parameters Papp, a concrete scenario SZc _i and at least one evaluation parameter Pval _j . The evaluation parameters Pval can, for example, relate to a maximum or minimum jerk, an average deceleration or acceleration, a minimum time period until a hypothetical collision of an ego vehicle with a target vehicle.

The application parameters Papp are stored in the application parameter database 320 and the evaluation parameters Pval in the evaluation database 340. The test agent 220 has access to the application parameter database 320 and the evaluation database 340. Within the scope of the invention, it can be provided that further databases are also used for additional information.

The test cases T generated in this way are stored in the test database 300. The data for a test case T _i can also contain a test identification number (test ID), a user name, a creation time and the name for the selected concrete scenario SZc _i .

To carry out a simulation of the created test cases T, the test agent 220 passes a test case T ₁ or several test cases T to the simulation module 400. The simulation module 400 comprises various sub-modules 410, 420, 430, which are designed to simulate specific properties of the test cases T for the driving task to be tested and can interact with each other.

They are designed as exchangeable sub-modules and have the task of simulating individual aspects of a respective test case T _i or a plurality of test cases T . A first sub-module 410 can be designed as an environment model module that relates to the environment and maps various environments of a motor vehicle 10. A second sub-module 420 can be designed as a driver model module that relates to the driving mode or driving style of a driver, such as dynamic or defensive. A third sub-module 430 can be designed as a vehicle model module that includes various models of a motor vehicle, for example with regard to the different design of the drive train, the driving dynamics and other sub-functions. The sub-modules 410, 420, 430 implement the description of the respective test case T _t for a specific situation or configuration and provide the necessary input data for the simulation of a driver assistance function in a driving function module 440.

The simulation module 400 or the various sub-modules 410, 420, 430 are connected to one or more databases 480 in which sensor signals from sensors 470 and other data are stored. The sensor signals are in particular measurement data of properties and characteristics of a motor vehicle 10 and of objects and events in the environment of the motor vehicle 10, which were recorded when driving along a route during a defined time window Δt _i and/or a defined route section Δx _i . The objects in the traffic environment of the motor vehicle 10 are in particular other road users such as motor vehicles, pedestrians, cyclists, and the events are, for example, an acceleration process, a lane change, or a traffic light switching. The sensors 470 can be designed in particular as radar systems, LIDAR systems for optical distance and speed measurement, image-recording 2D/3D cameras in the visible, IR and/or UV range, and/or GPS systems. Furthermore, acceleration sensors, speed sensors, capacitive sensors, inductive sensors, voltage sensors, torque sensors, precipitation sensors and/or temperature sensors, etc. can be designed.

The database 480 can also store additional historical data in the form of images, graphics, time series, parameters, etc. For example, target variables and target values that define a security standard for the simulation can be stored in the database 480. The database 480 can also be integrated into a cloud computing infrastructure.

The test cases T selected by the test agent 220 are integrated into the various sub-modules 410, 420, 430, with input signals such as sensor data, information about the vehicle state and driver inputs being provided by the sub-modules 410, 420, 430 in order to carry out a simulation of the selected test cases T. For this purpose, the sub-modules 410, 420, 430 use simulation algorithms such as X-in-the-Loop (XiL). In particular, the modeling of the sensors 470 is of particular importance, since inaccuracies and measurement errors in the sensor signals of the sensors 470 used must be taken into account when adapting the application parameters Papp. If a selected test case T _i or a plurality of test cases T cannot cover a specified driving task, further data is provided by means of the sub-modules 410, 420, 430 in order to be able to map and thus simulate a holistic driving event.

The simulation module 400 thus carries out a simulation of the test cases T for a specified driving task and passes the simulation results 450 to the evaluation module 500.

The evaluation module 500 has a driving function evaluation module 510, which evaluates the simulation results 450 with regard to the performance and functionality of one or more functions or the overall performance of the ADAS/ADS system, in particular in the form of key performance indicators (KPIs) for the various evaluation parameters Pval. In addition, the evaluation module 500 can include a simulation evaluation module 520 to determine the quality of the simulation process carried out, in particular in the form of simulation quality criteria (SQCs). From these evaluations, the evaluation module 500 creates evaluation results 550 for the various evaluation parameters Pval.

KPIs are used to describe the performance of an ADAS/ADS under test, with different KPIs being set for different evaluation categories such as comfort, safety, naturalness of driving and efficiency. In addition, further KPIs can be implemented to verify the correct functionality of the ADAS/ADS system under test. An example of a KPI is the evaluation of a minimum distance to another vehicle or an average acceleration in a deceleration scenario. The KPIs and SQCs can be represented by numerical values or by Boolean values.

Direct and indirect evaluation metrics can be used for both the KPIs and SQCs. Direct evaluation metrics contain data directly from simulated measurement data. Indirect evaluation metrics use scenario parameters in particular as a data source. An example of the application of indirect evaluation metrics is the implementation of simulation models that quantify and evaluate the subjective perception of a driver when carrying out a driver assistance function of an ADAS/ADS system. Based on the KPIs and SQCs, the results of a simulated test case T _i can be represented by numerical values or Boolean values.

The output module 700 uses the evaluation results 550, particularly in the form of the key performance indicators (KPIs), to create a sensitivity analysis 750 for the selected data sets of the application parameters Papp. The sensitivity analysis 750 contains an evaluation of the simulations and simulation results carried out for the selected test cases T. For this purpose, the output module 700 has a suitably designed software application. In particular, the sensitivity analysis 750 can be presented in the form of diagrams and histograms, which, for example, contain a representation of the changes and gradients of the KPIs and evaluation parameters Pval for each change to an application parameter Papp _i in each selected concrete scenario SZ _i . In addition, further evaluations such as averaging of the changes to the application parameters Papp in relation to all selected concrete scenarios SZc are conceivable. An application guide created automatically in this way based on the sensitivity analysis 750 documents the impacts and effects of the changes to the various application parameters Papp in the simulations carried out on the evaluation parameters Pval. This allows an application engineer to fine-tune the Papp application parameters in a more targeted and efficient manner.

It is obvious that the quality of the sensitivity analysis 750 and thus of the application guide created depends on the quality of the scenario description and the type of simulation as well as the definition of the evaluation metrics. Since the simulation module 400 comprises various sub-modules 410, 420, 430, a simple adaptation to different ADAS/ADS systems can be carried out by exchanging sub-modules. Furthermore, the sub-modules 410, 420, 430 can be further developed and validated independently of one another, which leads to a higher quality of the overall process. The simulation module 400 can therefore be adapted to different use cases by means of the exchangeable sub-modules 410, 420, 430. Different driving tasks can be implemented simply by changing the test strategy 230 of the test agent 220.

In 2 The procedural steps for the sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) are presented.

In a step S10, data records of application parameters Papp are generated.

In a step S20, test cases T are created by a test agent using a test strategy, whereby the test strategy selects at least one concrete scenario SZc _i and a data set of application parameters Papp for each test case T _i .

In a step S30, one or more evaluation parameters Pval are assigned to a test case T _i .

In a step S40, a simulation is carried out by a simulation module 400 with the created test cases T to determine simulation results 450.

In a step S50, an evaluation of the simulation results 450 is carried out by an evaluation module 500 for determining evaluation results 550.

In a step S60, a sensitivity analysis 750 of the selected data sets of the application parameters Papp for the ADAS/ADS system is generated and output by the output module 700, in particular in the form of diagrams and histograms for an application guide.

3 schematically illustrates a computer program product 900 comprising an executable program code 950 configured to carry out the method according to the first aspect of the present invention.

The present invention can be used to carry out a sensitivity analysis with regard to the effects of changes in application parameters for an ADAS/ADS system. This makes the effects and effects on the driving behavior of an ADAS/ADS system due to changes in application parameters transparent and understandable. The virtual simulation allows the effects of changes in application parameters to be calculated quickly and efficiently. In particular, diagrams and histograms can be created as part of an application guide that reflect the effect of successively changing application parameters and thus represent an important decision-making aid for an application engineer. In a further development, it can also be provided that the method and system according to the invention is used to generate an optimal data set of application parameters for an ADAS/ADS system. Overall, resources can be saved because both the actual driving of test routes with standard traffic situations and with specific corner cases can be reduced.

Reference symbols

10

Motor vehicle

100

system

200

Test agent module

220

Test agent

230

Test strategy

300

Test database

320

Application parameter database

330

Scenario database

340

Rating database

400

Simulation module

410

Environment model module

420

Driver model module

430

Vehicle model module

440

Driving function module

450

Simulation results

470

Sensors

480

Database

500

Evaluation module

510

Driving function evaluation module

520

Simulation evaluation module

550

Evaluation results

700

Output module

750

Sensitivity analysis

900

Computer program product

950

Program code

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 102019126195 A1 [0006]
• DE 102021200927 A1 [0007]
• DE 10308314 A1 [0008]
• DE 102019212830 A1 [0009]

Claims (15)

Method for sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) in at least one concrete scenario (SZc _i ), in particular for testing and training the ADAS/ADS system, wherein a concrete scenario (SZc _i ) represents a traffic event in a temporal sequence, comprising: - generating (S10) data sets of application parameters (Papp) to be tested; - creating (S20) test cases (T) by means of a test strategy (230) by a test agent (220), wherein the test strategy (230) selects at least one concrete scenario (SZc _i ) and one data set of application parameters (Papp _i ) for each test case (T _i ); - Assigning (S30) one or more evaluation parameters (Pval) to a test case (T _i ) - Carrying out (S40) a simulation by the simulation module (400) with the created test cases (T) to determine simulation results (450); - Carrying out (S50) an evaluation of the simulation results (450) by an evaluation module (500) to determine evaluation results (550); - Generating and outputting (S60) a sensitivity analysis (750) of the selected data sets of the application parameters (Papp) for the ADAS/ADS system by the output module (700), in particular in the form of diagrams and histograms for an application guide. Procedure according to Claim 1 , wherein the test cases (T) are stored in a test database (300), the application parameters (Papp) in an application parameter database (320), the concrete scenarios (SZc) in a scenario database (330), and the evaluation parameters (Pval) in an evaluation database (340). Procedure according to Claim 1 or 2 , wherein the test strategy (230) and/or the test agent (220) uses/use at least one software application with calculation methods and/or artificial intelligence algorithms for changing the application parameters (Papp). Procedure according to Claim 3 , where the algorithms and calculation methods are designed as mean values, minimum and maximum values, lookup tables, models of expected values, linear regression methods, Gaussian processes, fast Fourier transformations, integral and differential calculus, Markov methods, probability methods such as Monte Carlo methods, temporal difference learning, extended Kalman filters, radial basis functions, data fields, convergent neural networks, deep neural networks, and/or feedback neural networks. Method according to one of the Claims 1 until 4 , wherein the simulation module (400) has exchangeable sub-modules (410, 420, 430), wherein a first sub-module (410) is designed as an environment model module, a second sub-module (420) as a driver model module and a third sub-module (430) as a vehicle model module. Method according to one of the Claims 1 until 5 , wherein the simulation module (400) and/or the sub-modules (410, 420, 430) is/are connected to sensors (470) and/or a database (450) in order to obtain further information for the creation of simulation models, wherein the simulation models are passed on to a driving function module (440) for carrying out the simulation of a driver assistance function. Method according to one of the Claims 1 until 6 , wherein the evaluation module (500) comprises a driving function evaluation module (510) for determining the performance and safety of a driving function using key performance indicators (KPIs) and/or a simulation evaluation module (520) for determining the quality of the simulation using simulation quality criteria (SQCs), and wherein the evaluation results (550) comprise the key performance indicators (KPIs) and/or the simulation quality criteria (SQCs). Method according to one of the Claims 1 until 7 , where an application parameter (Papp _i ) represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius and/or a number. Method according to one of the Claims 1 until 8th , wherein the sensors (470) are designed as radar systems, LIDAR systems for optical distance and speed measurement, image-recording 2D/3D cameras in the visible, IR and/or UV range, GPS systems, acceleration sensors, speed sensors, capacitive sensors, inductive sensors, voltage sensors, torque sensors, precipitation sensors and/or temperature sensors. System (100) for sensitivity analysis of application parameters for automated driving functions of a driver assistance system (ADAS) and/or an automated driving system (ADS) in at least one specific scenario (SZc _i ), in particular for testing and training the ADAS/ADS system, wherein a specific scenario (SZc _i ) represents a traffic situation in a temporal Sequence, comprising a test module (200) with a test agent (220) and a test strategy (230), a simulation module (400), an evaluation module (500) and an output module (700), wherein the test agent (220) is designed to generate data sets of application parameters (Papp), to create test cases (T) by means of a test strategy (230), wherein the test strategy (230) selects at least one concrete scenario (SZc _i ) and a data set of application parameters (Papp _i ) for each test case (T _i ), and to assign one or more evaluation parameters (Pval) to a test case (T _i ); wherein the simulation module (400) is designed to carry out a simulation with the created test cases (T) to determine simulation results (450); wherein the evaluation module (500) is designed to carry out an evaluation of the simulation results (450) to determine evaluation results (550); and wherein the output module (700) is designed to generate and output a sensitivity analysis (750) of the selected data sets of the application parameters (Papp) for the ADAS/ADS system, in particular in the form of diagrams and histograms for an application guide. System (100) after Claim 10 , wherein the test cases (T) are stored in a test database (300), the application parameters (Papp) in an application parameter database (320), the concrete scenarios (SZc) in a scenario database (330), and the evaluation parameters (Pval) in an evaluation database (340). System (100) after Claim 10 or 11 , wherein the test strategy (230) and/or the test agent (220) uses/use at least one software application with calculation methods and/or artificial intelligence algorithms for changing the application parameters (Papp). System (100) according to one of the Claims 10 until 12 , where an application parameter (Papp _i ) represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius and/or a number. ADAS/ADS system for a motor vehicle (10), characterized in that for testing and training the ADAS/ADS system (210) the method according to one of the Claims 1 until 9 is used. Computer program product (900) comprising an executable program code (950) configured to perform the method according to one of the Claims 1 until 9 executes.

Images