DE102020005507A1 - Method for testing an automated driving function - Google Patents

Abstract

The invention relates to a method for testing an automated driving function, in particular an emergency braking function, of a vehicle. According to the invention, a confidence level for existing field tests and a confidence level for the driving function to be tested is determined over an entire effective range of an operational design domain, with measurement data from country-specific field tests being recorded and are stored centrally in a database, - then knowledge is extracted from this database by means of an event-driven time series analysis with a cluster formation, the extracted clusters and their parameter space defining a probability of occurrence of each logical scenario and a probability distribution of the associated parameters, - a sensitivity and reliability analysis Failure region identified in parameter space to predict a probability of failure for each logical scenario using sampling, with Verkeh rshotspotkatagorien of false-positive and false-negative events from various information sources can be transformed into driving situations characterized by cluster analysis with the help of a database for scenarios to be tested.

Description

The invention relates to a method for testing an automated driving function.

In the DE 10 2018 004 429 A1 The applicant, the full content of which is hereby incorporated by reference into this application, describes a method for testing a brake assistance system for a vehicle. A cluster-analytical characterization of driving situations is determined based on detected sensor signals to detect the surroundings and their system reactions when the vehicle is driving.

The invention is based on the object of specifying a method for testing an automated driving function which is improved over the prior art.

The object is achieved according to the invention by a method for testing an automated driving function having the features of claim 1.

In a method for testing an automated driving function, in particular an emergency braking function, a confidence level for existing field tests and a confidence level for the driving function to be tested are determined over the entire effective range of the so-called operational design domain. A device for performing the method comprises a computing module which is set up to determine a reliability for the automated driving function, in particular that of an automatic emergency braking system. For this purpose, measurement data from country-specific field tests are recorded and stored centrally in a database. Then the knowledge is extracted from this database by means of an event-driven time series analysis with a cluster formation. The extracted clusters and their parameter space define the probability of occurrence of each logical scenario and the probability distribution of the associated parameters. Then a sensitivity and reliability analysis identifies the failure area in the parameter space in order to predict the failure probability for each logical scenario with the help of sampling methods. Traffic hotspot categories of false-positive (sensor detects an object that does not exist in reality) and false-negative events (sensor does not recognize an object that exists in reality) from the various information sources in the driving situations characterized by cluster analysis using a database for the driving situations to be tested Transformed scenarios.

In addition, the method enables a holistic test decision basis for testing an automated driving function to be tested.

This provides a meaningful termination criterion for the testing, which is defined on the basis of the quality assessment of the optimization of the algorithm in the individual components and in the overall system.

Based on the prospective risk assessment, the termination criterion is based on various decisions. Either the function is safe enough and suitable for the customer, the function must be secured with further driving and simulation runs, or the function is not safe enough, then with a recommendation for further development of the automated function to be tested.

Embodiments of the invention are explained in more detail below with reference to drawings.

• 1 eine schematische Darstellung der Integration zwischen digitalen und physikalischen Erprobungen zur kundenorientierten Absicherung von automatischen Fahrfunktionen,
• 2 eine schematische Darstellung der Datenverarbeitungsstruktur des Datenerfassungssystems im Fahrzeug mit seinen Hauptelementen,
• 3 eine schematische Darstellung einer Korrelationsanalyse zwischen digitalen und physikalischen Erprobungen, sowie die Bestimmung $TT{C}_{E1}^{diff}$
  
  als quantifizierbarer Fehlerindikator von Umfeldwahrnehmung, $TT{C}_{E1}^{diff}$
  
  ist das Zeitintervall zwischen $TT{C}_{E1}^{HIL}\text{ und }TT{C}_{E1}^{FOT}$
  
  und
• 4 eine schematische Darstellung der Zeitreihen für die Erfassungsdaten aus dem Erfassungssystem für ein logisches Szenario mit einem sich vor dem Fahrzeug befindlichen, stationären Objekt in einer linken Kurveneingangssituation, auch als Verbundkurve nach links bezeichnet,
• 5 eine schematische Darstellung eines logischen Szenarios in einer Ontologie mit einem sich vor dem Fahrzeug befindlichen, stationären Objekt in einer linken Kurveneingangssituation, auch als Verbundkurve nach links bezeichnet,
• 6 eine schematische Darstellung der kumulativen Verteilungsfunktion CDF und Wahrscheinlichkeitsdichtefunktion PDF für die Zufallsvariable $TT{C}_{E1}^{min}\left[\text{s}\right]$
  
  mit einem Parameter a = -0.318, Parameter b = 1.90614 und Parameter c = 1.9614 als Ausgang für den Zufallgenerator der gemeinsamen Dichte der Zufallsvariablen der vom Ersatzmodell zu selektierenden Daten aus dem Cluster C₁,
• 7 eine schematische Darstellung der kumulativen Verteilungsfunktion CDF und Wahrscheinlichkeitsdichtefunktion PDF für die Zufallsvariable $TT{C}_{E1}^{diff}\left[\text{s}\right]$
  
  mit folgenden Parametern (µ = -0.451483, σ = 0.36633, a = -1.45, b = 0.143) als Eingang für den Zufallsgenerator der gemeinsamen Dichte der Zufallsvariablen der vom Ersatzmodell zu selektierenden Daten aus dem Cluster C₁ und die Zufallsvariable υ_x [km/h] mit folgenden Paramatern (a = 7.94651, b = 59.5984, c = 83.3891),
• 8 eine schematische Darstellung der kumulativen Verteilungsfunktion CDF und Wahrscheinlichkeitsdichtefunktion PDF für die Zufallsvariable K_ego[1/km] mit folgenden Parametern (µ = 0.268827, σ = 1.1418, a = -4.5, b = 3.88) als Eingang für den Zufallsgenerator der gemeinsamen Dichte der Zufallsvariablen der vom Ersatzmodell zu selektierenden Daten aus dem Cluster C₁ und die Zufallsvariable d_y[m] mit folgenden Parametern (µ = -0.2987, σ = 0.564, a = - 2.6, b = 0.8),
• 9 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^3\triangleq \left(TT{C}_{E1}^{min}\le 0.5\text{ s}\right)$
  
  mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1000 mit Standardfehler e_{P̅ f} = 0.0094 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.1),
• 10 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^2\triangleq \left(0.5s<TT{C}_{E1}^{min}\le 1s\right)$
  
  mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1800 mit Standardfehler e_{P̅ f} = 0.0092 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.19),
• 11 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^1\triangleq \left(1s<TT{C}_{E1}^{min}\le 2s\right)$
  
  mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 500 mit Standardfehler e_P̅f= 0.022 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.51),
• 12 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^0\triangleq \left(2s<TT{C}_{E1}^{min}\right)$
  
  mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1600 mit Standardfehlere_{P̅ f} = 0.01 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.2),
• 13 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^3\triangleq \left(TT{C}_{E1}^{min}\le 0.5\text{ s}\right)$
  
  mit Hilfe Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as = 800 mit Standardfehler e_{P̅ f} = 0.0088 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.1),
• 14 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^2\triangleq \left(0.5s<TT{C}_{E1}^{min}\le 1s\right)$
  
  mit Hilfe Adaptiven Samplings mit 5 Iterationen (Stichprobenzahl N_as = 900 mit Standardfehler e_{P̅ f} = 0.0088 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.18),
• 15 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^1\triangleq \left(1s<TT{C}_{E1}^{min}\le 2s\right)$
  
  mit Hilfe Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as= 800 mit Standardfehler e_{P̅ f} = 0.036 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.52), und
• 16 eine schematische Darstellung der geschätzten Versagenswahrscheinlichkeit der Schwere der Ereignisse $S_{C1}^0\triangleq \left(2s<TT{C}_{E1}^{min}\right)$
  
  Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as = 800 mit Standardfehler e_{P̅ f} = 0.084 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.26).

Show:

• 1 a schematic representation of the integration between digital and physical tests for customer-oriented protection of automatic driving functions,
• 2 a schematic representation of the data processing structure of the data acquisition system in the vehicle with its main elements,
• 3 a schematic representation of a correlation analysis between digital and physical tests, as well as the determination $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}$
  
  as a quantifiable error indicator of environmental perception, $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}$
  
  is the time interval between $TT{\mathrm{C.}}_{\mathrm{E.}1}^{H\mathrm{I.}\mathrm{L.}}\text{and}TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}$
  
  and
• 4th a schematic representation of the time series for the acquisition data from the acquisition system for a logical scenario with a stationary object in front of the vehicle in a left curve entry situation, also referred to as a composite curve to the left,
• 5 a schematic representation of a logical scenario in an ontology with a stationary object in front of the vehicle in a left corner entry situation, also referred to as a compound curve to the left,
• 6th a schematic representation of the cumulative distribution function CDF and probability density function PDF for the random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$
  
  with one parameter a = -0.318, parameter b = 1.90614 and parameters c = 1.9614 as output for the Random generator of the common density of the random variables of the data to be selected by the substitute model from cluster C ₁ ,
• 7th a schematic representation of the cumulative distribution function CDF and probability density function PDF for the random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right]$
  
  with the following parameters (µ = -0.451483, σ = 0.36633, a = -1.45, b = 0.143) as input for the random generator of the common density of the random variables of the data to be selected by the substitute model from the cluster C ₁ and the random variable υ _x [km / h] with the following parameters (a = 7.94651, b = 59.5984, c = 83.3891),
• 8th a schematic representation of the cumulative distribution function CDF and probability density function PDF for the random variable K _ego [1 / km] with the following parameters (µ = 0.268827, σ = 1.1418, a = -4.5, b = 3.88) as input for the random generator of the common density of the random variables of the data to be selected by the substitute model from the cluster C ₁ and the random variable d _y [m] with the following parameters (µ = -0.2987, σ = 0.564, a = - 2.6, b = 0.8),
• 9 a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 0.5\text{s}\right)$
  
  using Monte Carlo simulation (sample number N _mc = 1000 with standard error e _{P̅ f} = 0.0094 and estimated failure probability P̅ _f = 0.1),
• 10 a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 1s\right)$
  
  using Monte Carlo simulation (sample number N _mc = 1800 with standard error e _{P̅ f} = 0.0092 and estimated failure probability P̅ _f = 0.19),
• 11 a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 2s\right)$
  
  with the help of Monte Carlo simulation (number of samples N _mc = 500 with standard error e _P̅f = 0.022 and estimated failure probability P̅ _f = 0.51),
• 12th a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq \left(2s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)$
  
  with the help of Monte Carlo simulation (number of samples N _mc = 1600 with standard _{error P̅ f} = 0.01 and estimated failure probability P̅ _f = 0.2),
• 13th a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 0.5\text{s}\right)$
  
  using adaptive sampling with 4 iterations (sample number N _as = 800 with standard error e _{P̅ f} = 0.0088 and estimated failure probability P̅ _f = 0.1),
• 14th a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 1s\right)$
  
  using adaptive sampling with 5 iterations (sample number N _as = 900 with standard error e _{P̅ f} = 0.0088 and estimated failure probability P̅ _f = 0.18),
• 15th a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 2s\right)$
  
  using adaptive sampling with 4 iterations (sample number N _as = 800 with standard error e _{P̅ f} = 0.036 and estimated failure probability P̅ _f = 0.52), and
• 16 a schematic representation of the estimated probability of failure of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq \left(2s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)$
  
  Adaptive sampling with 4 iterations (sample number N _as = 800 with standard error e _{P̅ f} = 0.084 and estimated failure probability P̅ _f = 0.26).

Corresponding parts are provided with the same reference symbols in all figures.

Based on 1 to 16 a method for testing an automated driving function, in particular an emergency braking function, of a vehicle is described below.

• Alle automatisierten Fahrfunktionen müssen die ISO Norm 26262 zur funktionalen Sicherheit erfüllen und dürfen sich nicht gegeneinander negativ beeinflussen.

The starting point for this procedure is as follows:

• All automated driving functions must meet ISO standard 26262 for functional safety and must not negatively influence one another.

When using automated driving functions, making safe decisions and planning collision-free trajectories play a decisive role.

In order to be able to ensure the requirements of functional safety according to ISO 26262 for an automatic driving function, in particular for an emergency braking function, and in particular to safely prevent an unwanted automatic braking intervention in an automatic emergency braking system, the functions must be adequately safeguarded by driving and simulation runs.

The hazard and risk analysis of the ISO standard 26262 defines the risk as a combination of the probability of the occurrence of damage and the severity of this damage, derived from the ISO 14971 standard for the application of risk management to medical devices.

In the DE 10 2018 004 429 A1 , the complete content of which is hereby incorporated by reference, a method and a device for testing a brake assistance system for a vehicle, in particular for a truck, are described. In this method, a cluster-analytical characterization of driving situations is determined based on detected sensor signals for the detection of the surroundings and their system reactions when the vehicle is driving.

Based on this, in the method described here for testing an automated driving function, in particular an emergency braking function, a confidence level for existing field tests and a confidence level for the driving function to be tested is determined over the entire effective range of the so-called operational design domain. A device for performing the method comprises a computing module that is set up to determine a reliability for the automated driving function, in particular that of an automatic emergency braking system. For this purpose, measurement data from country-specific field tests are recorded and stored centrally in a database. Then the knowledge is extracted from this database by means of an event-driven time series analysis with a cluster formation. The extracted clusters and their parameter space define the probability of occurrence of each logical scenario and the probability distribution of the associated parameters. Then a sensitivity and reliability analysis identifies the failure area in the parameter space in order to predict the failure probability for each logical scenario with the help of sampling methods. Traffic hotspot categories of false-positive (sensor detects an object that does not exist in reality) and false-negative events (sensor does not recognize an object that exists in reality) from the various information sources in the driving situations characterized by cluster analysis using a database for the driving situations to be tested Transformed scenarios.

In addition, the method enables a holistic test decision basis for testing an automated driving function to be tested.

This provides a meaningful termination criterion for the testing, which is defined on the basis of the quality assessment of the optimization of the algorithm in the individual components and in the overall system.

Based on the prospective risk assessment, the termination criterion is based on various decisions. Either the function is safe enough and suitable for the customer, the function must be secured with further driving and simulation runs, or the function is not safe enough, then with a recommendation for further development of the automated function to be tested.

1 shows a schematic representation of the integration of digital tests 1 and physical tests 2 for customer-oriented safeguarding of automatic driving functions.

For this purpose, a knowledge-based test method 3 , comprehensive digital testing 1 , in particular a requirements-based testing, and a data-driven test method 4th , comprehensive physical testing 2 , in particular scenario-based testing.

In the process, process steps I to VIII, for example, are carried out in a mandatory manner, the order of which can be changed.

Process step I: Exclusion of logical errors through verification of the automatic driving function with the help of a so-called hardware-in-the-loop platform in accordance with the knowledge-based requirements.

The logical error represents an error in the algorithm according to knowledge-based design.

Method step II: Identification of statistical errors in an automated driving function through scenario-based testing, in particular for an emergency braking function, with the help of a so-called cluster analysis in a cloud database.

The statistical error represents an error in operation due to environmental influences.

The warning events of an existing assistance system, in particular a brake assistance system, are called disengagement events, i.e. H. Shutdown events assumed for a shutdown criterion in the autonomous mode of a future automated driving function.

The cluster analysis is a procedure with which cases (pedestrians, objects) can be grouped on the basis of given criteria. The groups found in this way - also called clusters - then each contain cases that are similar. The cases in different clusters, however, differ more.

Method step III: Identification of criticality metrics for the driving function to be tested through a correlation analysis between the criticality threshold from synthetic driving scenarios and events from the individually grouped naturalistic driving situations.

Execution of simulation calculations for defined test scenarios on the hardware-in-the-loop platform, and derivation of the criticality threshold from the simulation results using a suitable regression function. A criticality metric is, for example, the time interval between the vehicle in front and the vehicle behind in order to assess the criticality of a given situation. Process step IV: test execution on the hardware-in-the-loop platform for the individually grouped logical scenarios with what is known as ontology-based scenario management 5 , including in particular a partitioning of equivalence classes.

An ontology is a knowledge model that is used in knowledge management in the so-called Semantic Web to provide knowledge structures. This information, its properties and relationships to one another are formally represented and can be read by machines and humans.

The ontology with the help of the characteristic signal curves from the individual grouped logical scenarios uses the conversion rules between acquisition data with an open control loop and control data with a closed control loop in order to expand the test coverage over the entire effective range of the so-called operational design domain. To such an operational design domain coverage 6th belongs in addition to ontology-based scenario management 5 also an exploration of the parameter space described below 7th , including a correlation and sensitivity analysis.

Method step V: exploration of the parameter space 7th for the individually grouped logical scenarios with the help of the so-called sensitivity analysis in order to generate an approximation model for the parameters under consideration.

The sensitivity analysis deals with the propagation of uncertainties in the generation of the approximation model for the parameters under consideration.

The task consists in the systematic quantification of the influences that model parameters with uncertainties have on the outputs of a model.

The parameter sensitivities allow, among other things, to draw conclusions about the robustness of the model prognoses with regard to disturbances and thus provide a measure of confidence for the results obtained.

Process step VI: Estimation of the probability of failure for the individual grouped logical scenarios. Determination of the failure area in the parameter space. Prediction of the probability of security through prospective risk assessment for measurable security with the help of a so-called reliability analysis.

Definition of the curve of the tolerable risk as a confidence level for estimating the probability of exceeding the safety margin using different sampling methods.

There is thus a measurable safety assessment 8th with reference values of the risk acceptance threshold 9 , including a reliability analysis.

That in this measurable safety assessment 8th The diagram shown for the risk assessment matrix has a severity SG labeled x-axis that ranges from "minor" to "catastrophic" and one with frequency H labeled y-axis, which ranges from “inconceivable” to “frequent”.

The risk acceptance threshold 10 is by a line from the top left (high frequency, ie high frequency H , low severity SG ) to the bottom right (low frequency, ie low frequency H , catastrophic severity SG ) accepted. The area below this line indicates an acceptable risk VR and the area above this line indicates an unacceptable risk NVR .

In the event of an unacceptable risk, ie an unacceptable risk NVR , the residual risk should be below the risk acceptance threshold 10 by optimizing the driving function to be tested using fallback solutions with the so-called Minimal Risk Condition (MRC) and adapting the ASIL classification of the component concerned over the entire effective range of the so-called operational design domain.

Process step VII: Definition and application of the test termination criteria on the basis of the generated tolerable risk curve as a reference safety threshold for further development of the driving function to be tested 11 .

The risk assessment matrix is made by combining the probability of occurrence of harm and the severity SG created this damage.

Extraction of the decision-making basis for risk-minimizing measures, whether more test kilometers are required, more simulations are to be carried out or a further development of the automated function to be tested is necessary.

• Verfahrensschritt VIII: Vergleich zusätzlicher Felddaten mit logischen Szenarien, Gegebenenfalls Erweiterung der Anzahl der Cluster für die Gruppierung, Wiederholdung und Anwendung der Verfahrensschritte II bis VII.

Another procedure for taking additional field data into account in the case of characteristic courses of the system input variables that have already been determined provides, for example, as optional process steps:

• Process step VIII: Comparison of additional field data with logical scenarios, if necessary, expansion of the number of clusters for grouping, repetition and application of process steps II to VII.

The method is explained below using an exemplary embodiment.

The braking assistance system for performing the automated driving function, in particular the emergency braking function, comprises a detection unit for detecting a distance between the vehicle and an object located in front of the vehicle.

The detection unit comprises a radar sensor, by means of which a redundant measurement of the distance is carried out based on a radar signal transit time and a differential speed or relative speed between the preceding or stationary object and the vehicle based on a frequency shift. In addition, an image acquisition unit, in particular a camera, is provided, by means of which the surroundings of the vehicle are recorded.

Objects, roads, peripheral buildings, lane marking lines and road boundaries, such as delineator posts and guardrails, as well as traffic signs are determined from the captured images and corresponding environmental data and environmental parameters are generated and forwarded to a control unit.

Furthermore, the emergency braking function is carried out in several escalation levels E1 to E3. In this driving function, an optical and / or acoustic warning is issued as a warning in a first escalation level E1, automatic partial braking is carried out as a haptic warning in a second escalation level E3, and full braking is carried out as a braking process in a third escalation level E3. An automatic emergency braking process to prevent the vehicle from colliding with the object in front of the vehicle must be triggered if the distance, the relative speed, the acceleration of the vehicle and the acceleration of the vehicle in front are related to one another in a certain way.

Incorrect triggering of the emergency braking function is avoided or at least significantly reduced by detecting peripheral buildings or peripheral objects on the road, such as delineator posts, crash barriers and traffic signs. Such peripheral buildings and peripheral objects are usually dependent on the type of road and are also taken into account by adapting the trigger release condition to the classification of the road when the driving function is triggered.

In process step II, the measurement data from country-specific field tests, also known as Field Operational Tests (FOT), are recorded. After the central storage of the measurement data in a database, there is a clustering, ie a grouping, of natural driving scenarios based on brake warning events of escalation level E1 of a brake assistance function. The hierarchical cluster analysis of an event-based analysis is carried out with signal clusters C ₁ with the compound curve to the left, C ₂ with the turning line to the right, C ₃ with the turning line to the left and C ₄ with the compound curve to the right, as in FIG DE 10 2018 004 429 A1 is described.

2 shows a schematic representation of the data processing structure of the data acquisition system in the vehicle with its main elements, ie it shows a measurement technology and data analysis concept for the worldwide safeguarding of automated driving functions, which is scalable for recording vehicle sensor and / or environment sensor data of specific driving events on the one hand and permanent data for statistical validation and software development on the other hand allowed.

The data processing structure includes event-based data acquisition 12th with a measurement technology for field operational test data 13th a corner case detection, especially for scenario elicitation 14th with a data ingest station 15th , especially for scenario extraction, and cloud-based data storage 16 with a scenario database 17th , especially for re-simulation and scenario fuzzing.

The measurement data from country-specific field tests are recorded during scenario-based testing and stored centrally in a cloud database.

This requires higher capacities for data acquisition and storage, but has the advantage that a complete re-simulation with modified functional software versions of all field tests (Field Operational Test - FOT) is possible.

Formula (1) describes the time TTC _E1 until the collision for escalation level E1 as a disengagement event for the switch-off criterion in the autonomous mode of an emergency braking function in future automated vehicles, if the accelerations of the vehicle and the vehicle ahead or at a standstill are constant over time, as follows: $$TT{\mathrm{C.}}_{\mathrm{E.}1}=\frac{d_{\mathrm{E.}1}}{v_x},\forall v_x>0$$

It is υ _x the longitudinal speed.

bis zur Kollision mit einem sich vor dem Fahrzeug befindlichen stationären Objekt in der Eskalationsstufe E1 ausgeführt wird. Das Fahrzeug bewegt sich mit einer Längsgeschwindigkeit υ_x[km/h]. In der Simulation bewegt sich das Fahrzeug mit einer Längsgeschwindigkeit $v_x^{HiL}\left[km/h\right].$

In process step III of the process, simulation calculations for defined test scenarios are carried out on the hardware-in-the-loop platform. A driving scenario with a straight road is provided, which is used to determine the time $TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}\left[s\right]$

until it collides with a stationary object in front of the vehicle in escalation level E1. The vehicle moves at a longitudinal speed of υ _x [km / h]. In the simulation, the vehicle moves at a longitudinal speed $v_x^{Hi\mathrm{L.}}\left[km/H\right].$

Der Parameter P3 ist der Wendepunkt, bei dem 50% $TT{C}_{E1}^{HiL}$

zwischen minimalen und maximalen Wert erreicht sind. Der Parameter P4 steuert die Steigung, wobei größere Werte eine flachere Kurve erzeugen. $$TT{C}_{E1}^{HiL}=P1+\frac{P2-P1}{1+{\left(10\right)}^{\left(P3-v_x^{HiL}\right)\ast P4}},\forall v_x^{HiL}>0$$

Formula (2) describes the regression function using a dose-response curve with four parameters. In pharmacology, dose-effect curves graphically describe the relationship between the administered dose of an active ingredient and its effect. Logistic regression is therefore a dose-response model, which is also referred to as a sigmoid dose-response relationship. As always, P1 and P2 denote the minimum and maximum levels of the $TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}.$

The parameter P3 is the turning point at which 50% $TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}$

between the minimum and maximum value are reached. The parameter P4 controls the slope, with larger values producing a flatter curve. $$TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}=\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">P</figure-callout>}1+\frac{P2-\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">P</figure-callout>}1}{1+{\left(10\right)}^{\left(P3-v_x^{Hi\mathrm{L.}}\right)\ast P\mathrm{4th}}},\forall v_x^{Hi\mathrm{L.}}>0$$

3 shows a schematic representation of a correlation analysis between digital and physical tests 1, 2. A legend is shown above the diagram in order to assign the markings shown for the hardware-in-the-loop platform HiL and the markings shown for the field operational tests FOT can. 3 shows schematically a time interval with a stationary object in front of the vehicle with a different initial relative speed for the escalation level E1 in a scenario with a straight road that is used on a hardware-in-the-loop platform. In addition, a theoretical ideal curve of escalation level E1 versus ego speed is assumed. For example, the 179 false trips from cluster C ₁ with the compound curve to the left, as in FIG DE 10 2018 004 429 A1 is used for the correlation analysis.

und $TT{C}_{E1}^{FOT}$

berechnet. $${\rho }_{TT{C}_{E1}^{HiL},TT{C}_{E1}^{FOT}}=\frac{{\displaystyle {\sum }_{i=1}^n\left(TT{C}_{E1}^{HiL}-{\overline{TTC}}_{E1}^{HiL}\right)\left(TT{C}_{E1}^{FOT}-{\overline{TTC}}_{E1}^{FOT}\right)}}{\sqrt{{\left(TT{C}_{E1}^{HiL}-{\overline{TTC}}_{E1}^{HiL}\right)}^2\cdot {\left(TT{C}_{E1}^{FOT}-{\overline{TTC}}_{E1}^{FOT}\right)}^2}}$$

In the correlation analysis, the correlation coefficients according to Pearson's product-moment correlation are used as a measure of the linear relationship between two variables $TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}$

and $TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}$

calculated. $${\rho }_{TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}},TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}}=\frac{{\displaystyle {\sum }_{i=1}^n\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}-{\overline{TT\mathrm{C.}}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}\right)\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}-{\overline{TT\mathrm{C.}}}_{\mathrm{E.}1}^{\mathrm{F.}OT}\right)}}{\sqrt{{\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}-{\overline{TT\mathrm{C.}}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}\right)}^2\cdot {\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}-{\overline{TT\mathrm{C.}}}_{\mathrm{E.}1}^{\mathrm{F.}OT}\right)}^2}}$$

The sigmoid dose-response relationship as a regression function is calculated with the following parameters P1 = 0.52, P2 = 4.25, P3 = 37.6, P4 = 0.02.

Der Korrelationskoeffizient kann Werte zwischen -1 und 1 annehmen, wobei ein Korrelationskoeffizient von 0 bedeutet, dass kein Zusammenhang zwischen beiden Variablen existiert. Ein Korrelationskoeffizient von +1 beschreibt einen perfekten positiven Zusammenhang zwischen beiden Variablen, während eine Korrelation von -1 eine perfekte Antikorrelation beschreibt.The correlation coefficient is calculated between the ideal curve of escalation level E1 and the 179 false alarms from cluster C ₁ with the compound curve to the left, as in DE 10 2018 004 429 A1 is described so that ${\rho }_{TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}},TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}}=\mathrm{0.8055.}$

The correlation coefficient can have values between -1 and 1, whereby a correlation coefficient of 0 means that there is no connection between the two variables. A correlation coefficient of +1 describes a perfect positive relationship between the two variables, while a correlation of -1 describes a perfect anti-correlation.

4th shows a schematic representation of the time series for the acquisition data from the acquisition system for a logical scenario with a stationary object in front of the vehicle in a left curve entry situation, also referred to as a composite curve to the left.

5 shows a schematic representation of a logical scenario in an ontology with a stationary object in front of the vehicle in a left curve entry situation, also referred to as a composite curve to the left. The vehicle is referred to here as an ego vehicle.

In method step VI, the ontology is used with the aid of the characteristic signal curves from the logical scenario.

In software engineering, an ontology is defined as the explicit specification of a conceptualization.

Conceptualization is an abstract, simplified representation of some selected parts of an application domain.

Hence, the ontology can be viewed as a knowledge base consisting of terminological and assertional boxes.

While terminological boxes (TBoxes) describe the concepts of an application domain, these concepts are expressed through hierarchical classes, axioms and properties.

The assertional boxes (ABoxes), on the other hand, represent instances of classes and observed facts of situational knowledge.

The Web Ontology Language is implemented in the ontology for every logical scenario, which allows logical operators to be combined in rules. Invalid or forbidden combinations are therefore removed from the scenario catalog.

In the road design, the three mathematical functions (straight line, circular arc and clothoid) are used, whereby the elements should be joined together without kinks at the transition points.

The compound curve consists of a circular arc and an introductory and an exiting clothoid. The turning line consists of two composite curves with different directions of curvature.

des sich vor dem Fahrzeug befindlichen, stationären Objekts über die Zeit t [s] sind die Erfassungsdaten mit offenem Regelkreis. In 4 wird die relative Querablage $d_y^{rel}\left[m\right]$

als d_y[m] bezeichnet. ψ is the yaw rate of the ego vehicle.
The calculated curvature of the traveled route K _ego [1 / km] and the relative lateral offset $d_y^{rel}\left[m\right]$

of the stationary object in front of the vehicle over the time t [s] are the detection data with an open control loop. In 4th becomes the relative lateral offset $d_y^{rel}\left[m\right]$

referred to as d _y [m].

The acquisition data are converted according to the conversion rules as closed-loop control data in order to generate the specific scenarios with the entire parameter space.

The specific scenarios to be created expand the test coverage over the entire effective range of the so-called operational design domain in order to ensure the further development of the driving function 11 to be tested.

4th represents an ontology for a logical scenario from cluster C ₁ with 179 false events as 53.12% out of a total of 337 false events in the cluster analysis, as in DE 10 2018 004 429 A1 is described.

For example, the ontology for the logical scenario “compound curve to the left” from cluster C _{1 contains} a “consists_of” instruction to model the elements of a road network layout with two lane classes and one class for a hard shoulder. The instructions "Neighbor lane right" and "Neighbor _lane left" are used to arrange road elements in relation to one another. The position instances are generated on the basis of a "position" relation for the ontology road elements.

The instructions "left_of", "right_of", "front_of", "rear_of" are used to arrange the position instances with logical argumentation. The instructions “drives_up” and “lies_up” are used to control the dynamic objects with different position instances. The object "Obj1" is defined as a stationary object on the hard shoulder.

The parameter spaces and also relevant distribution functions are identified so that specific scenarios can be derived from these logical scenarios. Individual tests (concrete scenarios) are then carried out based on the parameterization.

das Cluster C₁ und das Cluster C₄ präsentiert, wird die Wahrscheinlichkeit durch $P\left(F_s^1|F_T\right)$

bestimmt.The test method requires the probability P _T that an emergency braking function will execute a functional scenario catalog U. Because the functional scenario $\text{"}{\mathrm{F.}}_s^1=\text{Compound curve "}$

the cluster C ₁ and the cluster C ₄ presented, the probability is through $P\left({\mathrm{F.}}_s^1|{\mathrm{F.}}_T\right)$

certainly.

das Cluster C₂ und das Cluster C₃ präsentiert, wird die Wahrscheinlichkeit durch P_t(F_tWendelinie) bestimmt. $$P_T=P\left(F_s^1|F_T\right)\ast P\left(F_s^1\right)+P\left(F_s^2|F_T\right)\ast P\left(F_s^2\right)$$

Because the functional scenario $\text{"}{\mathrm{F.}}_s^2=\text{Turning line "}$

the cluster C ₂ and the cluster C ₃ presented, the probability is determined by P _t (F _t turning line). $$P_T=P\left({\mathrm{F.}}_s^1|{\mathrm{F.}}_T\right)\ast P\left({\mathrm{F.}}_s^1\right)+P\left({\mathrm{F.}}_s^2|{\mathrm{F.}}_T\right)\ast P\left({\mathrm{F.}}_s^2\right)$$

definiert werden.In general, a functional scenario can be seen as a combination of an equivalence class for each dimension $U=\left[{\mathrm{F.}}_s^1,{\mathrm{F.}}_s^2,...,{\mathrm{F.}}_s^n\right]$

To be defined.

ergibt.The total probability of functional scenarios can generally be formulated for the dimensions n, whereby the number of scenarios is derived from the equivalence classes of the dimension ${\mathrm{\Pi }}_{i=1}^{U}i$

results.

mit einem Parameter a = -0.318, Parameter b = 1.90614 und Parameter c = 1.9614 als Ausgang für den Zufallsgenerator der gemeinsamen Dichte der Zufallsvariablen der vom Ersatzmodell zu selektierenden Daten aus dem Cluster C₁. Die Auswertung des Ersatzmodells wird für die Definition der Grenzzustandsfunktion verwendet. 6th shows a schematic representation of the cumulative distribution function CDF and probability density function PDF (in the 6th to 8th shown, the markings can be identified using the legend above the diagrams) for the random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

with a parameter a = -0.318, parameter b = 1.90614 and parameter c = 1.9614 as output for the random generator of the common density of the random variables of the data to be selected by the substitute model from cluster C ₁ . The evaluation of the substitute model is used to define the limit state function.

mit folgenden Parametern (µ = -0.451483, σ = 0.36633, a = -1.45, b = 0.143) als Eingang für den Zufallsgenerator der gemeinsamen Dichte der Zufallsvariablen der vom Ersatzmodell zu selektierenden Daten aus dem Cluster C₁ und die Zufallsvariable υ_x [km/h] mit folgenden Paramatern (a = 7.94651, b = 59.5984, c = 83.3891). Die Auswertung des Ersatzmodells wird für die Definition der Grenzzustandsfunktion verwendet. 7th shows a schematic representation of the cumulative distribution function CDF and probability density function PDF for the random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right]$

with the following parameters (µ = -0.451483, σ = 0.36633, a = -1.45, b = 0.143) as input for the random generator of the common density of the random variables of the data to be selected by the substitute model from the cluster C ₁ and the random variable υ _x [km / h] with the following parameters (a = 7.94651, b = 59.5984, c = 83.3891). The evaluation of the substitute model is used to define the limit state function.

8th shows a schematic representation of the cumulative distribution function CDF and probability density function PDF for the random variable K _{ego [} 1 / km] with the following parameters (µ = 0.268827, σ = 1.1418, a = -4.5, b = 3.88) as input for the random generator of the common density of the random variables of the data to be selected by the substitute model from the cluster C ₁ and the random variable d _y [m] with the following parameters (µ = -0.2987, σ = 0.564, a = -2.6, b = 0.8). The evaluation of the substitute model is used to define the limit state function.

mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1000 mit Standardfehler e_{P̅ f} = 0.0094 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.1). Die Markierungen sind in der Legende darüber erklärt. In 9 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des kritischen Schweregrads $\overline{P_f}\left(S_{C1}^3|C_1\right)$

mit Monte-Carlo Simulation gezeigt, wobei die Stichproben des Versagens im Vergleich zu den Stichproben des Überlebens sehr gering sind und sich am linken Ende der Figur befinden, was zeigt, dass das Ziel für die niedrigeren Werte von υ_x [km/h] getroffen ist. Das Ziel wird für die Werte von υ_x höher als 50 [km/h] verfehlt. Die Abbruchkriterien der Simulation hängen vom Vergleich zwischen dem Standardfehler der Überschreitungswahrscheinlichkeit und dem geforderten Standardfehler δ_{P̅ f} = 0.05 ab. Aus der Stichprobenzahl von N_mc = 1000 sind die Stichprobenzahl $N_{mc}^{Versagen}=99$

für das Versagen und die Stichprobenzahl $N_{mc}^{\text{Ü}berleben}=901$

für das Überleben ausgewertet. 9 shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 0.5\text{s}\right)$

using Monte Carlo simulation (sample number N _mc = 1000 with standard error e _{P̅ f} = 0.0094 and estimated failure probability P̅ _f = 0.1). The markings are explained in the legend above. In 9 becomes the expected value of the estimator for the failure probability of the critical severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right)$

shown with Monte Carlo simulation, where the samples of failure are very small compared to the samples of survival and are located at the left end of the figure, showing that the target hit for the lower values of υ _x [km / h] is. The goal is for the values of υ _x higher than 50 [km / h] missed. The termination criteria of the simulation depend on the comparison between the standard error of the exceedance _probability and the required standard error δ _{P̅ f} = 0.05. The number of samples is derived from the sample number of N _mc = 1000 $N_{mc}^{VersaGen}=99$

for the failure and the number of samples $N_{mc}^{\text{Ü}berleben}=901$

evaluated for survival.

mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1800 mit Standardfehler e_P̅f= 0.0092 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.19). Die Markierungen sind in der Legende darüber erklärt. In 10 wird der Erwartungswert des Schätzer für die Versagenswahrscheinlichkeit des höheren Schweregrads $\overline{P_f}\left(S_{C1}^2|C_1\right)$

mit Monte-Carlo Simulation gezeigt, wobei die Anzahl der Stichproben des Versagens erhöht wird. Das Ziel ist für die höheren Werte der relativen Längsgeschwindigkeit 1υ_x [km/h] eingetroffen. Dadurch wird eine leicht nach rechts verschobene Wolke der Stichproben des Versagens erzeugt. Aus der Stichprobenzahl von 1N_mc = 1800 sind die Stichprobenzahl $N_{mc}^{Versagen}=335$

für das Versagen und die Stichprobenzahl $N_{mc}^{\text{Ü}berleben}=1465$

für das Überleben ausgewertet. 10 shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 1s\right)$

using Monte Carlo simulation (sample number N _mc = 1800 with standard error e _P̅f = 0.0092 and estimated failure probability Pag _f = 0.19). The markings are explained in the legend above. In 10 becomes the expected value of the estimator for the probability of failure of the higher severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^2|{\mathrm{C.}}_1\right)$

shown with Monte Carlo simulation, increasing the number of samples of failure. The goal has been reached for the higher values of the relative longitudinal speed 1υ _x [km / h]. This creates a cloud of the samples of failure, shifted slightly to the right. From the sample number of 1N _mc = 1800 are the sample number $N_{mc}^{VersaGen}=335$

for the failure and the number of samples $N_{mc}^{\text{Ü}berleben}=1465$

evaluated for survival.

mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 500 mit Standardfehler e_{P̅ f} = 0.022 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.51). Die Markierungen sind in der Legende darüber erklärt. In 11 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des mittleren Schweregrads $\overline{P_f}\left(S_{C1}^1|C_1\right)$

mit Monte-Carlo Simulation gezeigt, wobei die Parameter für υ_x [km/h] sich für die Stichproben mit dem Versagen erhöhen und die Wolke der Stichproben für das Versagen sich wieder nach rechts bewegt. Aus der Stichprobenzahl von N_mc = 500 sind die Stichprobenzahl $N_{mc}^{Versagen}=257$

für das Versagen und die Stichprobenzahl $N_{mc}^{\text{Ü}berleben}=243$

für das Überleben ausgewertet. 11 shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 2s\right)$

using Monte Carlo simulation (sample number N _mc = 500 with standard error e _{P̅ f} = 0.022 and estimated failure probability P̅ _f = 0.51). The markings are explained in the legend above. In 11 becomes the expected value of the estimator for the probability of failure of the medium severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^1|{\mathrm{C.}}_1\right)$

shown with Monte Carlo simulation, where the parameters for υ _x [km / h] increase for the samples with failure and the cloud of samples for failure moves to the right again. From the sample number of N _mc = 500 are the sample number $N_{mc}^{VersaGen}=257$

for the failure and the number of samples $N_{mc}^{\text{Ü}berleben}=243$

evaluated for survival.

mit Hilfe Monte-Carlo-Simulation (Stichprobenzahl N_mc = 1600 mit Standardfehler e_{P̅ f} = 0.01 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.2). Die Markierungen sind in der Legende darüber erklärt. In 12 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des niedrigen Schweregrads $\overline{P_f}\left(S_{C1}^0|C_1\right)$

mit Monte-Carlo Simulation gezeigt, wobei die Wolke der Stichproben für das Versagen auf der rechten Seite liegt. Aus der Stichprobenzahl von N_mc = 1600 sind die Stichprobenzahl $N_{mc}^{Versagen}=321$

für das Versagen und die Stichprobenzahl $N_{mc}^{\text{Ü}berleben}=1279$

für das Überleben ausgewertet. 12th shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq \left(2s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)$

using Monte Carlo simulation (sample number N _mc = 1600 with standard error e _{P̅ f} = 0.01 and estimated failure probability P̅ _f = 0.2). The markings are explained in the legend above. In 12th becomes the expected value of the estimator for the probability of failure of the low severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^0|{\mathrm{C.}}_1\right)$

shown with Monte Carlo simulation with the cloud of samples for failure on the right. From the sample number of N _mc = 1600 are the sample number $N_{mc}^{VersaGen}=321$

for the failure and the number of samples $N_{mc}^{\text{Ü}berleben}=1279$

evaluated for survival.

mit Hilfe Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as = 800 mit Standardfehler e_{P̅ f} = 0.0088 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.1). Die Markierungen sind in der Legende darüber erklärt. In 13 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des kritischen Schweregrads $\overline{P_f}\left(S_{C1}^3|C_1\right)$

mit Adaptive-Importance Sampling gezeigt. Der Algorithmus für das Adaptive-Importance Sampling verwendet vier Iterationen, wobei jede Iteration aus 200 Stichproben besteht. Aus der Stichprobenzahl von N_as = 800 sind die Stichprobenzahl $N_{as}^{Versagen}=512$

für das Versagen und die Stichprobenzahl $N_{as}^{\text{Ü}berleben}=288$

für das Überleben ausgewertet. Der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit wird mit P̅_f = 0.1 berechnet und bestätigt somit das Ergebnis aus der Monte-Carlo Simulation beim gleichen Schweregrad $\overline{P_f}\left(S_{C1}^3|C_1\right).$

13th shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 0.5\text{s}\right)$

using adaptive sampling with 4 iterations (sample number N _as = 800 with standard error e _{P̅ f} = 0.0088 and estimated failure probability P̅ _f = 0.1). The markings are explained in the legend above. In 13th becomes the expected value of the estimator for the failure probability of the critical severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right)$

shown with Adaptive-Importance Sampling. The Adaptive-Importance Sampling algorithm uses four iterations, with each iteration consisting of 200 samples. From the sample number of N _as = 800 are the sample number $N_{as}^{VersaGen}=512$

for the failure and the number of samples $N_{as}^{\text{Ü}berleben}=288$

evaluated for survival. The expected value of the estimator for the probability of failure is calculated with P̅ _f = 0.1 and thus confirms the result from the Monte Carlo simulation with the same degree of severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right).$

mit Hilfe Adaptiven Samplings mit 5 Iterationen (Stichprobenzahl N_as = 900 mit Standardfehler e_{P̅ f} = 0.0088 und geschätzter Versagenswahrscheinlichkeit P̅_f= 0.18). Die Markierungen sind in der Legende darüber erklärt. In 14 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des höheren Schweregrads $\overline{P_f}\left(S_{C1}^2|C_1\right)$

mit Adaptive Importance Sampling gezeigt, wobei die Stichproben in fünf Iterationen durchgeführt sind. Die ersten drei Iterationen werden mit dem Budget von 100 Stichproben, die vierte mit 200 Stichproben und die fünfte mit 400 Stichproben durchgeführt. Aus der Stichprobenzahl von N_as = 900 sind die Stichprobenzahl $N_{as}^{Versagen}=434$

für das Versagen und die Stichprobenzahl $N_{as}^{\text{Ü}berleben}=446$

für das Überleben ausgewertet. Dies zeigt, dass die Varianz mit Adaptive-Importance Sampling erfolgreich eliminiert wird. 14th shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 1s\right)$

using adaptive sampling with 5 iterations (sample number N _as = 900 with standard error e _{P̅ f} = 0.0088 and estimated failure probability P̅ _f = 0.18). The markings are explained in the legend above. In 14th becomes the expected value of the estimator for the probability of failure of the higher severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^2|{\mathrm{C.}}_1\right)$

with Adaptive Importance Sampling with the sampling performed in five iterations. The first three iterations are performed with the budget of 100 samples, the fourth with 200 samples, and the fifth with 400 samples. From the sample number of N _as = 900 are the sample number $N_{as}^{VersaGen}=434$

for the failure and the number of samples $N_{as}^{\text{Ü}berleben}=446$

evaluated for survival. This shows that the variance is successfully eliminated with Adaptive-Importance Sampling.

mit Hilfe Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as = 800 mit Standardfehler e_{P̅ f} = 0.036 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.52). Die Markierungen sind in der Legende darüber erklärt. In 15 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des mittleren Schweregrads $\overline{P_f}\left(S_{C1}^1|C_1\right)$

mit Adaptive Importance Sampling gezeigt. Die Stichproben werden in vier Iterationen durchgeführt, wobei die ersten drei Iterationen mit einem Budget von 100 Stichproben und die vierte mit 500 Stichproben durchgeführt wird. Aus der Stichprobenzahl von N_as = 800 sind die Stichprobenzahl $N_{as}^{Versagen}=616$

für das Versagen und die Stichprobenzahl $N_{as}^{\text{Ü}berleben}=184$

für das Überleben ausgewertet. 15th shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 2s\right)$

using adaptive sampling with 4 iterations (number of samples N _as = 800 with standard error e _{P̅ f} = 0.036 and estimated failure probability P̅ _f = 0.52). The markings are explained in the legend above. In 15th becomes the expected value of the estimator for the probability of failure of the medium severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^1|{\mathrm{C.}}_1\right)$

shown with Adaptive Importance Sampling. The samples are performed in four iterations, with the first three iterations being performed with a budget of 100 samples and the fourth with 500 samples. From the sample number of N _as = 800 is the sample number $N_{as}^{VersaGen}=616$

for the failure and the number of samples $N_{as}^{\text{Ü}berleben}=184$

evaluated for survival.

mit Hilfe Adaptiven Samplings mit 4 Iterationen (Stichprobenzahl N_as = 800 mit Standardfehler e_P̅f= 0.084 und geschätzter Versagenswahrscheinlichkeit P̅_f = 0.26). Die Markierungen sind in der Legende darüber erklärt. In 16 wird der Erwartungswert des Schätzers für die Versagenswahrscheinlichkeit des niedrigen Schweregrads $\overline{P_f}\left(S_{C1}^0|C_1\right)$

mit Adaptive Importance Sampling gezeigt. Aus der Stichprobenzahl von N_as = 800 ist die Stichprobenzahl $N_{as}^{Versagen}=581$

für das Versagen und die Stichprobenzahl $N_{as}^{\text{Ü}berleben}=219$

für das Überlegen ausgewertet. Die Adaptive-Importance Sampling-Methode erzeugt für den Schweregrad eine statistische Unsicherheit mit dem Ergebnis der Versagenswahrscheinlichkeit. 16 shows a schematic representation of the estimated failure probability of the severity of the events ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq \left(2s>TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)$

using adaptive sampling with 4 iterations (number of samples N _as = 800 with standard error e _P̅f = 0.084 and estimated failure probability P̅ _f = 0.26). The markings are explained in the legend above. In 16 becomes the expected value of the estimator for the probability of failure of the low severity $\overline{P_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^0|{\mathrm{C.}}_1\right)$

shown with Adaptive Importance Sampling. From the sample number of N _as = 800 is the sample number $N_{as}^{VersaGen}=581$

for the failure and the number of samples $N_{as}^{\text{Ü}berleben}=219$

evaluated for deliberation. The Adaptive-Importance Sampling method generates a statistical uncertainty for the severity with the result of the probability of failure.

analysiert.The sensitivity analysis is presented as an example for the E ₁ events from cluster C ₁ . The required behavior models are created from the design of experiments as substitute models for the important parameters of the individual driving situations from cluster C ₁ . The required sensitivity measures are in relation to the response size $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

analyzed.

und d_x [m]) und dem Ausgang des Ersatzmodells $\left(TT{C}_{E1}^{min}\left[\text{s}\right]\right).$

In method step V, the sensitivity measures are determined with the aid of the sensitivity analysis, with the inputs of the substitute model $\text{(}TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right],v_x\left[\text{km}/\text{H}\right],k_{eGO}\left[\text{1}/\text{km}\right],d_y\left[\text{m}\right]$

and d _x [m]) and the output of the substitute model $\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]\right).$

geschlossen werden kann.These parameters should be used to examine whether the E ₁ event affects the critical value of the smallest time to collision $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

can be closed.

Some of the input parameters are strongly correlated. The undesired correlations between the input variables make modeling difficult. Therefore the variables (κ _ego [1 / km], d _y [m] and d _x [m]) are no longer taken into account for the substitute model.

The substitute models or metamodels represent an approximation of reality.

als Eingangsvariable des Zufallsgenerators der gemeinsamen Dichte der Zufallsvariablen mit Hilfe der abgeschnittenen Normalverteilung darzustellen ist und die Zufallsvariable υ_x [km/h] als Eingangsvariable des Zufallsgenerators der gemeinsamen Dichte der Zufallsvariablen mit abgeschnittener Normalverteilung darstellbar ist. Die Zufallsvariable $TT{C}_{E1}^{min}\left[\text{s}\right]$

ist als Ausgangsvariable des Zufallsgenerators der gemeinsamen Dichte der Zufallsvariablen mit Hilfe der Dreiecksverteilung darzustellen.The modeling of the random variables for the parameters of the cluster C ₁ is implemented in such a way that the random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right]$

is to be represented as the input variable of the random generator of the common density of the random variables with the help of the truncated normal distribution and the random variable υ _x [km / h] can be represented as the input variable of the random generator of the common density of the random variables with the truncated normal distribution. The random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

is to be represented as the output variable of the random generator of the common density of the random variables with the help of the triangular distribution.

resultiert aus $TT{C}_{E1}^{FOT}\left[\text{s}\right]\text{ und }TT{C}_{E1}^{HiL}\left[\text{s}\right]$

wie folgt: $$TT{C}_{E1}^{diff}=TT{C}_{E1}^{HiL}-TT{C}_{E1}^{FOT}$$

The random variable $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right]$

results from $TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}\left[\text{s}\right]\text{and}TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}\left[\text{s}\right]$

as follows: $$TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}=TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}-TT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}$$

While the probability density function PDF shows the distribution of the values, calculates the cumulative distribution function CDF the cumulative probability for a given value.

festgelegt. Dabei beschreibt $TT{C}_{E1}^{min}\left[\text{s}\right]$

den letzten Zeitpunkt, in dem die Umgebungssensoren, insbesondere der Radarsensor, das stationäre Objekt als relevant bewertet haben.In this case, the failure criterion to be considered is through $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

set. It describes $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

the last point in time at which the environmental sensors, in particular the radar sensor, rated the stationary object as relevant.

wird durch die Dreiecksverteilung modelliert, wobei die Wahrscheinlichkeitsdichtefunktion PDF der Dreieckverteilung drei Parametern hat: die Parameter a für die Untergrenze, b für die obere Grenze und c für den wahrscheinlichsten Wert. $$f_{PDF}\left(TT{C}_{E1}^{min}\right)=\{\begin{array}{cc}\frac{2\left(TT{C}_{E1}^{min}-a\right)}{\left(b-a\right)\left(c-a\right)},& wenna\le TT{C}_{E1}^{min}\le c\\ \frac{2\left(b-TT{C}_{E1}^{min}\right)}{\left(b-a\right)\left(b-c\right)},& wennc<TT{C}_{E1}^{min}\le b\end{array}$$

$$f_{PDF}\left(TT{C}_{E1}^{min}\right)=\{\begin{array}{rr}\hfill \frac{{\left(TT{C}_{E1}^{min}-a\right)}^2}{\left(b-a\right)\left(c-a\right)},& \hfill wenna\le TT{C}_{E1}^{min}\le c\\ \hfill 1-\frac{{\left(b-TT{C}_{E1}^{min}\right)}^2}{\left(b-a\right)\left(b-c\right)},& \hfill wennc<TT{C}_{E1}^{min}\le b\end{array}$$

The output variable of the substitute model $TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\left[\text{s}\right]$

is modeled by the triangular distribution, where the probability density function PDF the triangular distribution has three parameters: the parameters a for the lower limit, b for the upper limit and c for the most likely value. $$f_{P\mathrm{D.}\mathrm{F.}}\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)=\{\begin{array}{cc}\frac{2\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}-a\right)}{\left(b-a\right)\left(c-a\right)},& wenna\le TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le c\\ \frac{2\left(b-TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)}{\left(b-a\right)\left(b-c\right)},& wennc<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le b\end{array}$$

$$f_{P\mathrm{D.}\mathrm{F.}}\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)=\{\begin{array}{rr}\hfill \frac{{\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}-a\right)}^2}{\left(b-a\right)\left(c-a\right)},& \hfill wenna\le TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le c\\ \hfill 1-\frac{{\left(b-TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\right)}^2}{\left(b-a\right)\left(b-c\right)},& \hfill wennc<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le b\end{array}$$

wird durch die abgeschnittene Normalverteilung modelliert, wobei die Wahrscheinlichkeitsdichtefunktion PDF der abgeschnittenen Normalverteilung vier Parametern hat: die Lageparameter µ, die Formparameter σ, die Parameter a für die Untergrenze, b für die obere Grenze. Die Funktion φ ist die Dichtefunktion bei der Normalverteilung. Die Funktion Φ ist die kumulative Verteilung bei der Normalverteilung. $$f_{PDF}\left(TT{C}_{E1}^{diff}\right)=\frac{1}{\sigma }\frac{\phi \left(\frac{TT{C}_{E1}^{diff}-\mu }{\sigma }\right)}{\mathrm{\Phi }\left(\frac{b-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)},\forall a\le TT{C}_{E1}^{diff}\le b$$

$$f_{CDF}\left(TT{C}_{E1}^{diff}\right)=\frac{1}{\sigma }\frac{\mathrm{\Phi }\left(\frac{TT{C}_{E1}^{diff}-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)}{\mathrm{\Phi }\left(\frac{b-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)},\forall a\le TT{C}_{E1}^{diff}\le b$$

The output variable of the random generator $TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\left[\text{s}\right]$

is modeled by the truncated normal distribution, where the probability density function PDF the truncated normal distribution has four parameters: the position parameters µ , the shape parameters σ , the parameters a for the lower limit, b for the upper limit. The function φ is the density function in the normal distribution. The function Φ is the cumulative distribution in the normal distribution. $$f_{P\mathrm{D.}\mathrm{F.}}\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\right)=\frac{1}{\sigma }\frac{\phi \left(\frac{TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}-\mu }{\sigma }\right)}{\mathrm{\Phi }\left(\frac{b-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)},\forall a\le TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\le b$$

$$f_{\mathrm{C.}\mathrm{D.}\mathrm{F.}}\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\right)=\frac{1}{\sigma }\frac{\mathrm{\Phi }\left(\frac{TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)}{\mathrm{\Phi }\left(\frac{b-\mu }{\sigma }\right)-\mathrm{\Phi }\left(\frac{a-\mu }{\sigma }\right)},\forall a\le TT{\mathrm{C.}}_{\mathrm{E.}1}^{diff}\le b$$

ISO 26262 contains a hazard analysis and risk assessment to determine the required ASIL (Automotive Safety Integrity Level) and to assess the potential risks of electrical and / or electronic malfunctions that can violate the safety goals. The risk-oriented approach classifies the risk R for each potentially dangerous driving situation as follows: $$\mathrm{R.}\approx {\displaystyle \sum _0^{H\in H}{\mathrm{S.}}_H\ast {\mathrm{E.}}_H\ast {\mathrm{C.}}_H}$$

This defines the risk classification with the following two impact factors, where H corresponds to the number of dangerous driving situations.

Severity of the effect of a dangerous driving situation h, where ${\mathrm{S.}}_H\in \left\{{\mathrm{<figure-callout\; id="0"\; label="S.\; H"\; filenames="DE102020005507A1\_0140.png,DE102020005507A1\_0141.png"\; state="\{\{state\}\}">S.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="1"\; label="S.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">S.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="2"\; label="S.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">S.</figure-callout>}}_H^{},{\mathrm{S.}}_H^3\right\}.$

Probability of exposure to a dangerous driving situation h, where ${\mathrm{E.}}_H\in \left\{{\mathrm{<figure-callout\; id="0"\; label="E.\; H"\; filenames="DE102020005507A1\_0140.png,DE102020005507A1\_0141.png"\; state="\{\{state\}\}">E.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="1"\; label="E.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">E.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="2"\; label="E.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">E.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="3"\; label="E.\; H"\; filenames="DE102020005507A1\_0140.png,DE102020005507A1\_0141.png"\; state="\{\{state\}\}">E.</figure-callout>}}_H^{},{\mathrm{E.}}_H^{\mathrm{4th}}\right\}.$

Controllability level in the respective driving situation h, where ${\mathrm{C.}}_H\in \left\{{\mathrm{<figure-callout\; id="0"\; label="C.\; H"\; filenames="DE102020005507A1\_0140.png,DE102020005507A1\_0141.png"\; state="\{\{state\}\}">C.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="1"\; label="C.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">C.</figure-callout>}}_H^{},{\mathrm{<figure-callout\; id="2"\; label="C.\; H"\; filenames="DE102020005507A1\_0139.png,DE102020005507A1\_0140.png"\; state="\{\{state\}\}">C.</figure-callout>}}_H^{},{\mathrm{C.}}_H^3\right\}.$

für die automatisierte Fahrfunktion mit Automatisierungsstufen ohne Eingriff des Fahrers zugeordnet werden, wenn die zu testende Fahrfunktion schwer oder nicht kontrollierbar ist.The ASIL determination can determine the controllability level ${\mathrm{C.}}_H^{}$${}_3^{}$

can be assigned for the automated driving function with automation levels without driver intervention if the driving function to be tested is difficult or impossible to control.

In the so-called “shadow mode”, data is collected from customer vehicles on public roads in order to offer an efficient validation method for autonomous driving compared to a small number of autonomous test vehicles on certain routes.

The termination criteria are defined by specifying the reference values for the risk assessment matrix on the basis of today's driving functions using the example of an emergency braking assistant, which is used for the further development of the automated driving function to be tested in connection with the expansion of the entire effective range of the so-called operational design domain and / or increasing the automation level will.

The termination criteria represent the test decision bases that can be used in connection with existing regulations and framework conditions for a possible customer approval. Otherwise, the customer approval in this application is more of a rhetorical question, as the information from the field and from simulations continuously supplement the risk management.

The reliability analysis takes place with the help of the modeling of the substitute model from the sensitivity analysis by defining the limit state function.

For reliability analyzes using substitute models by determining the most important input variables, the models must be capable of forecasting in order to reliably calculate the scatter and thus the failure probabilities.

Its complement, the failure probability, is usually calculated as a measure of safety.

The failure in this context does not necessarily mean the total collapse of the driving function.

Every impermissible state of the driving function is referred to as a failure, so the probability of failure is the probability of such a state occurring.

The random variables are to be determined by the distribution type and the corresponding distribution parameters.

The failure state is defined by a deterministic limit state function.

The limit state function g (Z) is the answer of the substitute model under consideration to a realization of the n basic variables, which is scaled so that $$G\left(x\right)=\{\begin{array}{c}\le \gamma ,beiVersaGen\\ >\gamma ,beiÜberleben\end{array}$$

The predefined safety margin is γ, where the limit state function g (Z) is the relationship between resistance and load as a function of Z, where Z describes a vector of all the uncertainty variables that represent the loads and resistances.

From formula (10) it can be seen that the limit of the failure range, i.e. the transition from the safe to the failed state, is marked by g (Z) = γ.

der Raum der n Basisvariablen, dann ist der Versagensbereich der Unterraum ${\mathbb{Q}}_f\subseteq {ℝ}^n,$

welcher definiert ist durch $${\mathbb{Q}}_f=\left\{z\in {ℝ}^n|g\left(z\right)\le \gamma \right\}$$

Be ${ℝ}^n$

the space of the n base variables, then the failure region is the subspace ${\mathbb{Q}}_f\subseteq {ℝ}^n,$

which is defined by $${\mathbb{Q}}_f=\left\{z\in {ℝ}^n|G\left(z\right)\le \gamma \right\}$$

wie in Formel (12) dargestellt. Die Versagenswahrscheinlichkeit einer gemeinsamen Dichtefunktion wird mit P_f bezeichnet und ist gegeben durch $$P_f=P\left[Z\in {\mathbb{Q}}_f\right]=P\left[g\left(Z\right)\le \gamma \right]={\displaystyle \int \stackrel{{\mathbb{Q}}_f}{\overbrace{\cdots }}}{\displaystyle \int }{f}_z\left(z\right)dz$$

With formulas (10) and (11), the failure probability is the integral of the common density function of the random variable f _z (z) over the failure range ${\mathbb{Q}}_f,$

as shown in formula (12). The failure probability of a common density function is denoted by P _f and is given by $$P_f=P\left[Z\in {\mathbb{Q}}_f\right]=P\left[G\left(Z\right)\le \gamma \right]={\displaystyle \int \stackrel{{\mathbb{Q}}_f}{\overbrace{\cdots }}}{\displaystyle \int }{f}_z\left(z\right)dz$$

The function f _z (z) is the common density function of the base variable Z.

The calculation of expected values is an important problem for risk assessment.

An analytical calculation of the expected values with increasing dimensionality of the observed quantities is no longer practicable.

$\left(TT{C}_{E1}^{min}>2s\right),$

, $S_{C1}^1\triangleq \left(1s<TT{C}_{E1}^{min}\le 2s\right),$

$S_{C1}^2\triangleq \left(0.5s<TT{C}_{E1}^{min}\le 1s\right)\text{ und }{S}_{C1}^3\triangleq \left(TT{C}_{E1}^{min}\le 0.5s\right)$

definiert sind. Die prospektive Risikobewertung $R\left(\frac{C_1}{km}\right)$

für nachfolgendende Fahrzeuge der automatisierten Fahrfunktion kann hinsichtlich des extrahierten Clusters C₁ aus den Feldtests mit der folgenden Formel berechnet werden: $$\begin{array}{l}R\left(\frac{C_1}{km}\right)\approx P_f\left(S_{C1}^3|C_1\right)*P\left(\frac{C_1}{km}\right)+P_f\left(S_{C1}^2|C_1\right)*P\left(\frac{C_1}{km}\right)+P_f\left(S_{C1}^1|C_1\right)*P\left(\frac{C_1}{km}\right)\\ {\text{ +P}}_{\text{f}}\left(S_{C1}^0|C_1\right)*P\left(\frac{C_1}{km}\right)\end{array}$$

The severity of the impact S _{c1 is divided} into four regions, where ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq$

$\left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}>2s\right),$

, ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 2s\right),$

${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 1s\right)\text{and}{\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}\le 0.5s\right)$

are defined. The prospective risk assessment $\mathrm{R.}\left(\frac{{\mathrm{C.}}_1}{km}\right)$

For subsequent vehicles of the automated driving function, the extracted cluster C ₁ from the field tests can be calculated using the following formula: $$\begin{array}{l}\mathrm{R.}\left(\frac{{\mathrm{C.}}_1}{km}\right)\approx P_f\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right)*P\left(\frac{{\mathrm{C.}}_1}{km}\right)+P_f\left({\mathrm{S.}}_{\mathrm{C.}1}^2|{\mathrm{C.}}_1\right)*P\left(\frac{{\mathrm{C.}}_1}{km}\right)+P_f\left({\mathrm{S.}}_{\mathrm{C.}1}^1|{\mathrm{C.}}_1\right)*P\left(\frac{{\mathrm{C.}}_1}{km}\right)\\ {\text{+ P}}_{\text{f}}\left({\mathrm{S.}}_{\mathrm{C.}1}^0|{\mathrm{C.}}_1\right)*P\left(\frac{{\mathrm{C.}}_1}{km}\right)\end{array}$$

The Monte Carlo integration approximates the expected values through simulation-based approximation.

The solution of the integral by simulating rare events is known as Monte Carlo simulation and adaptive sampling.

The Monte Carlo simulation is suitable for the numerical evaluation of multidimensional integrals. The failure probability is defined in formula (12) as an integral of the common density function of the basic variables over the failure range.

The solution of the integral by simulation is called Monte-Carlo integration.

The result of formula (12) is estimated in the statistical sense from an artificial random sample generated on the computer.

The Monte Carlo simulation is used to calculate the expected values.

Instead of calculating the true expected value, one calculates an estimator using a random sample.

The estimator is unbiased if its expected value is identical to the exact solution sought.

The confidence interval of the estimator depends on its variance.

The goal of a static estimation method is to find an unbiased estimator with minimal variance.

ergibt sich für die Versagenswahrscheinlichkeit: $$P_f={\displaystyle \int f_z\left(z\right)dz}={\displaystyle {\int }_{-\infty }^{\infty }\cdots {\displaystyle {\int }_{-\infty }^{\infty }I\left(g\left(z\right)\right)f_z}}\left(z\right)dz$$

Somit kann die Versagenswahrscheinlichkeit als Erwartungswert E̅ (oder E) der geschätzten Indikatorfunktion betrachtet werden: $${\overline{P}}_f=E\left[I\left(g\left(z\right)\right)\right]$$

With the introduction of an indicator function I $$\mathrm{I.}\left(G\left(z\right)\right)=\{\begin{array}{c}\mathrm{1,}wennG\left(z\right)\le \mathrm{<figure-callout\; id="0"\; label="\gamma "\; filenames="DE102020005507A1\_0140.png,DE102020005507A1\_0141.png"\; state="\{\{state\}\}">\gamma </figure-callout>}\\ \mathrm{0,}wennG\left(z\right)>\gamma \end{array}$$

the probability of failure results: $$P_f={\displaystyle \int f_z\left(z\right)dz}={\displaystyle {\int }_{-\infty }^{\infty }\cdots {\displaystyle {\int }_{-\infty }^{\infty }\mathrm{I.}\left(G\left(z\right)\right)f_z}}\left(z\right)dz$$

Thus, the failure probability can be taken as an expected value E̅ (or E) the estimated indicator function can be considered: $${\overline{P}}_f=\mathrm{E.}\left[\mathrm{I.}\left(G\left(z\right)\right)\right]$$

The statistical mean with N realizations of the base variable x _i is an unbiased estimator of the formula (16): $${\overline{P}}_f\approx \frac{1}{N}{\displaystyle \sum _{i=1}^N\mathrm{I.}\left(G\left(z_i\right)\right)}$$

Its variance is calculated using the fidelity to expectations. For an expected probability of failure and a given variance of the unbiased estimator, formula (17) results in the requirement for the number of realizations that are to be simulated. $$var\left[{\overline{P}}_f\right]\approx \frac{P_f-P_f^2}{N_s}$$

In applications, the standard error related to the number of the simulation is often given as a comparison value, which is defined as follows: $$e_{{\overline{P}}_f}=\sqrt{\frac{var\left[{\overline{P}}_f\right]}{N_s}}$$

The number required N _S the simulation results from the given variance of the estimator.

The so-called indicator functions for various criteria (limit state functions according to the severity of the impact of a dangerous driving situation) are evaluated from the random samples.

The criterion of the quality of the probability of failure estimated from the sample is again the standard error.

The associated failure probabilities and statistical errors (standard error or the standard deviation of the estimated probability) are calculated.

• o $S_C^{}$$1_0^{}\triangleq \left(TT{C}_{E1}^{min}>2s\right)$
  
• o $S_C^{}$$1_1^{}\triangleq \left(1s<TT{C}_{E1}^{min}<2s\right)$
  
• o $S_C^{}$$1_2^{}\triangleq \left(0.5s<TT{C}_{E1}^{min}<1s\right)$
  
• o $S_C^{}$$1_3^{}\triangleq \left(TT{C}_{E1}^{min}<0.5s\right)$
  

The failure criteria or limit state functions are defined as follows:

• O ${\mathrm{S.}}_{\mathrm{C.}1}^0\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}>2s\right)$
  
• O ${\mathrm{S.}}_{\mathrm{C.}1}^1\triangleq \left(1s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}<2s\right)$
  
• O ${\mathrm{S.}}_{\mathrm{C.}1}^2\triangleq \left(0.5s<TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}<1s\right)$
  
• O ${\mathrm{S.}}_{\mathrm{C.}1}^3\triangleq \left(TT{\mathrm{C.}}_{\mathrm{E.}1}^{min}<0.5s\right)$
  

und den Standardfehler $e_{{\overline{P}}_f}\left(S_{C1}^3|C1\right)$

für die Schwere eines ungewollten Warnereignises $S_{C1}^3$

aus dem Cluster C₁ mit Monte-Carlo-Simulation. 9 shows an example of the estimated probability of failure ${\overline{P}}_f\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right)$

and the standard error $e_{{\overline{P}}_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^3|\mathrm{C.}1\right)$

for the severity of an unwanted warning event ${\mathrm{S.}}_{\mathrm{C.}1}^3$

from cluster C ₁ with Monte Carlo simulation.

definiert. Der geforderte Standardfehler δ_P̅f = 0.05.The termination criteria are the simulation runs N _mc after the stable achievement of a standard error $e_{{\overline{P}}_f}\left({\mathrm{S.}}_{\mathrm{C.}1}^3|{\mathrm{C.}}_1\right)\le 0.05$

Are defined. The required standard error δ _P̅f = 0.05.

Adaptive Sampling is a method that greatly reduces the required sample size compared to Monte Carlo.

It belongs to the family of importance sampling methods, which aim to generate samples in the failure area with high probability densities.

A different distribution is used to generate the samples than was defined for the parameters.

This influence on the statistics is corrected internally.

Adaptive Sampling automatically adjusts the distribution for the simulation in several iterative steps by only performing a statistical evaluation of the random samples leading to failure from the previous iteration.

With the mean values determined in this way and the covariance matrix, a multi-dimensional normal distribution is defined for the simulation.

The aim of sampling according to importance is to determine the variance of the estimator _{P̅ f} to reduce and thereby the required number N _S to reduce simulations.

The samples are generated using a probability distribution.

The random numbers are generated according to a specific density function instead of a uniform distribution. For the simulation of the basic variable z _i , the original density function f _z (z) is not used, but a specific density function h _Y (y), so that many samples of the introduced basic variable Y fall into each area that has a high probability density.

The random variables Y are defined in the Z range of values.

The integral of formula (15) expanded then gives: $$P_f=={\displaystyle {\int }_{-\infty }^{\infty }\cdots {\displaystyle {\int }_{-\infty }^{\infty }\mathrm{I.}\left(G\left(y\right)\right)}}\frac{f_Z\left(y\right)}{H_Y\left(y\right)}{H}_Y\left(y\right)dy$$

The probability of failure is the expected value in relation to the density function h _Y (y). $$P_f=\mathrm{E.}\left[\mathrm{I.}\left(G\left(x\right)\right)\frac{f_X\left(x\right)}{H_Y\left(x\right)}\right]$$

The probability of failure is estimated by simulating samples according to the distribution h _Y (x).

By simulating samples y _i according to the distribution h _Y (y), the estimated value of P̅ _f is: $${\overline{P}}_f\approx \frac{1}{N_s}{\displaystyle \sum _{i=1}^{N_s}\mathrm{I.}\left(G\left(y_i\right)\right)}\frac{f_X\left(y_i\right)}{H_Y\left(y_i\right)}$$

The variance of the estimator is calculated as follows: $$var\left[{\overline{P}}_f\right]\approx {\left(e_{{\overline{P}}_f}\right)}^2\approx \mathrm{E.}\left[{\left({\overline{P}}_f-P_f\right)}^2\right]$$

The variance of the estimator becomes zero if the density function has: $$H_Y\left(y\right)=\mathrm{I.}\left(G\left(y\right)\right)\frac{f_z\left(y\right)}{P_f}$$

In the Monte Carlo simulation, samples are generated according to the specified density function. The ratio between what lies in the integration area and what lies outside then converges towards the value of the integral. With Adaptive Importance Sampling, bar samples are instead used according to original variables, but a new simulated density function in order to intentionally influence the statistics. The simulation density function is also the correction between the original and simulated density function.

After a first simulation, the statistical moments of the samples of the basic variables in the failure area are calculated.

$$E\left[Y{Y}^T\right]=E\left[Z{Z}^T|g\left(z\right)\le \gamma \right]$$

A possible approximation of the original density function, as mentioned in formula (24), can be achieved by calculating the mean value vector E [Y]. These statistical moments are the distribution parameters of the normally distributed random variable Y, which are used for the simulation in a next iteration step. $$\mathrm{E.}\left[Y\right]=\mathrm{E.}\left[Z|G\left(z\right)\le \gamma \right]$$

$$\mathrm{E.}\left[Y{Y}^T\right]=\mathrm{E.}\left[Z{Z}^T|G\left(z\right)\le \gamma \right]$$

The simulation density h _Y (x) can thus be determined adaptively in several repeated runs.

The indicator function, the probability of failure and the statistical estimation error are in the 13th to 16 representable.

In many cases, Adaptive Sampling requires fewer samples to achieve the required accuracy.

In all cases, relatively high probabilities are calculated.

Only with very low failure probabilities (in the order of magnitude of 10 ^-6 ) does the efficiency of adaptive sampling become clear.

nach $S_{C1}^3$

ab, während $S_{C1}^0$

mit einer kleineren Wahrscheinlichkeit als $S_{C1}^1$

auftritt.The probability of failure increases ${\mathrm{S.}}_{\mathrm{C.}1}^1$

after ${\mathrm{S.}}_{\mathrm{C.}1}^3$

from while ${\mathrm{S.}}_{\mathrm{C.}1}^0$

with a smaller probability than ${\mathrm{S.}}_{\mathrm{C.}1}^1$

occurs.

The 13th to 16 show a similar type of cloud movement for the sample of failure from bottom left to top right compared to the 9 to 12th . This is due to the distribution of the measurement data, which serve as support points for the common density function of the random variables.

The common density function of the random variables for the random generator can be imprecise in the case of a low support point density. The equivalent model is used to determine the limit state function.

The common density function of the random variables for the random generator is extrapolated from the area covered by support points.

The Monte Carlo simulation is used as a reference and compared with the adaptive sampling.

The behavior of the Monte Carlo integration is typically that, for a desired standard error 0.05, the required sample size increases the smaller the probability of failure.

A required standard error δ _P̅ is first used for the investigations _f of 0.1, which is generally a good confidence in the result.

If adaptive sampling fulfills this goal after a few steps, but the iterations differ greatly, the required standard error δ _{P̅f is} reduced to 0.05 in order to force more iterations.

The results of the Monte Carlo integration and adaptive sampling confirm each other.

It is a method for checking the quality of the optimization of driving functions using the example of an emergency brake assistant with all possible traffic situations over the entire effective range of the so-called operational design domain.

List of reference symbols

1

digital testing

2

physical testing

3

knowledge-based test method

4th

data-driven test method

5

ontology-based scenario management

6

Operational Design Domain Coverage

7th

Exploration of the parameter space

8th

measurable safety rating

9

Reference values of the risk acceptance threshold

10

Risk acceptance threshold

11

Further development of the driving function to be tested

12

event-based data acquisition

13

Measurement technology for field operational test data

14th

Corner case detection

15th

Data ingest station

16

Cloud-based data storage

17th

Scenario database

I to VIII

Process step

CDF

cumulative distribution function

PDF

Probability density function

d _y

relative cross storage

e _{P̅ f}

Standard error of the estimator of the probability of failure

d _x

relative longitudinal distance

ψ

Yaw rate

y

Margin of safety

E.

Expected value of a target indicator function

E̅

Expected value of an estimated indicator function

f _PDF

Function of density distribution

f _CDF

Cumulative distribution function

σ

Standard deviation

µ

Average

a

minimum value of the triangular distribution / truncated normal distribution

b

maximum value of the triangular distribution / truncated normal distribution

c

most likely value of the triangular distribution

ϕ

Probability density function of the normal distribution

Φ

cumulative distribution function of the normal distribution

Time to collision for escalation level E ₁ , which is measured from the field operational tests and implemented as real criticality values from cluster C ₁

H

frequency

Time to collision for escalation level E ₁ , which is calculated from the hardware-in-the-loop platform and maps an ideal criticality threshold.

κ _ego

predictive route

N _S

Sample number

N _as

Number of samples using adaptive sampling

N _mc

Number of samples using Monte Carlo Sampling

Sample number for failure using Monte-Carlo sampling

Sample number for survival using Monte-Carlo sampling

Sample number for failure using Adaptive Sampling

Sample number for survival using adaptive sampling

NVR

unacceptable risk

P

Probability of occurrence

P̅ _f

estimated probability of failure

P̅ _f

Probability of failure

SG

Severity

critical severity

high severity

moderate severity

low severity

t

time

TTC _E1

Time to collision for escalation level E ₁

Random variable / response variable (minimum time until the collision, where the sensors provide the relevant object list for calculating the time until the collision. This random variable is used as the response variable for the reliability analysis and for the limit state function.

Random variable (the time interval between $TT{\mathrm{C.}}_{\mathrm{E.}1}^{Hi\mathrm{L.}}undTT{\mathrm{C.}}_{\mathrm{E.}1}^{\mathrm{F.}OT}$

as a quantifiable error indicator of environmental perception

VR

reasonable risk

υ _x

relative longitudinal speed

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102018004429 A1 [0002, 0017, 0055, 0064, 0067, 0082]

Claims (1)

A method for testing an automated driving function, in particular an emergency braking function, of a vehicle, characterized in that a confidence for existing field tests and a confidence level for the driving function to be tested is determined over an entire effective range of an operational design domain, with measurement data from country-specific field tests being recorded and are stored centrally in a database, - then knowledge is extracted from this database by means of an event-driven time series analysis with a cluster formation, with extracted clusters and their parameter space defining a probability of occurrence of each logical scenario and a probability distribution of associated parameters, - a sensitivity and reliability analysis a failure range identified in the parameter space to predict a probability of failure for each logical scenario using sampling, being traffic hotspot categories of false-positive and false-negative events from various information sources can be transformed into driving situations characterized by cluster analysis with the help of a database for scenarios to be tested.

Images