CN113935143A

CN113935143A - Estimating collision probability by increasing severity level of autonomous vehicle

Info

Publication number: CN113935143A
Application number: CN202110803496.6A
Authority: CN
Inventors: 芒努斯·于伦哈马尔; 卡尔·桑登; 马吉德·霍桑德·瓦基勒扎德; 安德烈亚斯·法利科文; 约阿基姆·奥尔森; 基利昂·茨维格迈尔; 恰达什·乌拉斯
Original assignee: Zheneti
Current assignee: Zheneti
Priority date: 2020-07-14
Filing date: 2021-07-14
Publication date: 2022-01-14
Also published as: US20220019713A1; EP3940487A1; EP3940487B1

Abstract

The present disclosure relates to estimating collision probability with increasing severity levels of autonomous vehicles, and more particularly, to a computer-implemented method and processing system for estimating failure probability for different severity levels of Automatic Driving System (ADS) functionality in a virtual test environment. In more detail, embodiments of the present disclosure enable estimation of the probability of collisions of varying severity by utilizing a Limit State Function (LSF) that obtains increasingly negative or positive values after a collision (e.g., when TTC or PET is 0). This may be accomplished, for example, by defining a function that is more negative for more severe crash severity. The LSF may, for example, comprise a function of the delta speed at impact (i.e., a negative delta speed at impact). Being able to generate failure probabilities for different severity levels for a given ADS function may be beneficial in focusing development and validation activities on the most desirable areas/aspects of the system under test (ADS function under test).

Description

Estimating collision probability by increasing severity level of autonomous vehicle

Technical Field

The present disclosure relates to an Automatic Driving System (ADS) of a vehicle, and more particularly, to a method and system for estimating a collision probability through an increased severity level of an autonomous vehicle.

Background

During the last years, the development of autonomous vehicles has been dramatic and many different solutions are being developed. More and more modern vehicles have Advanced Driver Assistance Systems (ADAS) to increase vehicle safety and more generally road safety. ADAS, which may be represented, for example, by an adaptive cruise control ACC, a collision avoidance system, a front collision warning, etc., is an electronic system that may assist a vehicle driver while driving. To function as intended, the ADAS may rely on input from multiple data sources such as automotive imaging, LIDAR, radar, image processing, computer vision, and/or on-board networks.

Today, the development of ADAS and Autonomous Driving (AD) is ongoing in a number of different technical areas within these areas. ADAS and AD will refer herein to the general term Automatic Driving System (ADs), corresponding to all different levels of automation as defined, for example, by SAE J3016 levels (0-5) of driving automation.

Therefore, in the less distant future, the ADS scheme will be applied to modern vehicles to a greater extent. ADS can be interpreted as a complex combination of various components, as can be defined as a system in which the perception, decision making and operation of the vehicle is performed by electronic and mechanical means instead of the driver, and as an introduction of automation into road traffic. This includes knowledge of the vehicle, the destination's treatment, and the surrounding environment. While the automated system has control over the vehicle, it allows a human operator to leave all responsibilities for the system. ADS typically incorporates various sensors for sensing the surroundings of the vehicle, e.g., radar, LIDAR, sonar, video cameras, navigation systems such as GPS, odometers, and/or Inertial Measurement Units (IMUs), from which advanced control systems can interpret the sensed information to identify appropriate navigation paths as well as obstacles and/or related signs.

However, the ADS function as described above must be required to operate with high integrity to provide a sufficiently low risk to the vehicle occupants and their surroundings. Ensuring that the risk is low enough may require a large amount of data that is difficult to process for statistical proof, and according to an example will require, for example, about one hundred vehicles to drive for five centuries to obtain. Moreover, a key aspect of the verification of the autonomic function is to provide evidence that the ADS function complies with acceptable safety specifications. Safety codes are a statistical objective that generally specifies the highest acceptable failure frequency that results in an accident with a given severity level. In the automotive industry, the specification may be converted to an Automotive Safety Integrity Level (ASIL) using the safety goals of the ISO26262 standard based on an estimate of the severity, exposure, and controllability of the fault.

As an example, highest integrity (ASIL D) is sought for failures that are common (exposure level E4), difficult to control (controllability level C3), and may cause accidents leading to serious injury or death (severity level S3). Safety goals (i.e., requirements) set to ensure that such failures do not occur are therefore assigned ASIL D according to ISO 26262. This in effect means that the security target should have less than 10^-9Estimated failure rate per hour. For those associated with severity level S2 (likely survival of participants) or S1 (mild/moderate injury)Failures, given the same exposure level (E4) and controllability level (C3), result in a failure that can be converted to an acceptable failure rate of 10, respectively^-8And 10^-7Failed/hour ASIL C and ASIL B.

It is not a simple task to estimate whether complex autonomic functions meet safety specifications for different severity levels, and a brute-force approach may be said to prove infeasible (Kalra & Padock, 2016). There is therefore a need in the art for improvements in validating and developing autonomic functions in the automotive industry, and in particular for new solutions that are reliable and effective at the same time providing more detailed results for further analysis than are currently known.

Disclosure of Invention

It is therefore an object of the present disclosure to provide a computer-implemented method and corresponding computer-readable storage medium for estimating failure probability for different severity levels of ADS functionality of a vehicle to mitigate all or at least some of the disadvantages of the presently known approaches.

In particular, it is an object of the present disclosure to provide a method or tool for evaluating compliance of ADS functions to security specifications using statistical information to save resources and time required for verification and/or development activities.

These and other objects are achieved by means of a computer implemented method and a corresponding computer readable storage medium for estimating a failure probability for different severity levels of an ADS function of a vehicle and a control system as defined in the appended claims. The term "exemplary" should be understood in this context to serve as an example, instance, or illustration.

According to a first aspect of the present disclosure, a computer-implemented method for estimating failure probability for different severity levels of Automatic Driving System (ADS) functionality in a virtual test environment is provided. The method includes obtaining a parameterized statistical model indicating a statistical distribution in an Operational Design Domain (ODD) of the ADS function to be tested with respect to a plurality of scenes in the real-world environment. The method further includes performing a structural reliability method (e.g., subset) by running a statistical model and a Limit State Function (LSF) based on the parameterizationSimulation) to estimate a failure probability of the ADS function over time in the virtual test environment, wherein the LSF indicates a performance of the ADS function. In more detail, LSF g_i(θ) is the scene parameter set θ ═ θ₁,θ₂,…,θ_n]As a function of (c). LSF g_i(θ) includes a first function g_F(theta) and a second function g_S(theta). First function g_F(θ) is a function of at least one scene parameter indicative of a fault scene. Second function g_S(θ) is a function of at least one scenario parameter indicative of a severity level of the fault scenario, such that the estimated failure probability of the ADS function is further indicative of estimated failure probabilities for at least two different severity levels.

The resulting estimated failure probability can be used in order to assess compliance of the ADS function with the security specifications, so that advantages in reducing time and resources for validation activities are readily achievable. Moreover, the method proposed above can be used as a method for generating important test cases for development, thereby providing an advantage in terms of reducing time and resources for development.

According to a second aspect of the present disclosure, there is provided a (non-transitory) computer readable storage medium storing one or more programs configured for execution by one or more processors of a processing system, the one or more programs including instructions for performing the method according to any one of the embodiments disclosed herein. With this aspect of the disclosure, there are similar advantages and preferred functions as in the first aspect of the disclosure discussed above.

The term "non-transitory" as used herein is intended to describe a computer-readable storage medium (or "memory") that does not include a propagating electromagnetic signal, but is not intended to otherwise limit the type of physical computer-readable storage device included by the phrase "computer-readable medium or memory". For example, the terms "non-transitory computer readable medium" or "tangible memory" are intended to include types of storage devices that do not necessarily permanently store information, including, for example, Random Access Memory (RAM). Program instructions and data stored in a tangible computer accessible storage medium in a non-transitory form may further be transmitted by a transmission medium or a signal such as an electrical, electromagnetic, or digital signal that may be transmitted over a communication medium such as a network and/or a wireless link. Thus, with respect to limitations on data storage persistence, the term "non-transitory" as used herein is a limitation of the medium itself (i.e., tangible, not a signal) (e.g., RAM versus ROM).

According to a third aspect of the present disclosure, a control system for estimating failure probability for different severity levels of Automatic Driving System (ADS) functionality in a virtual test environment is provided. The control circuitry is configured to obtain a parameterized statistical model indicative of statistical distribution in an Operational Design Domain (ODD) of the ADS function with respect to a plurality of scenes in the real-world environment. Further, the control circuit is configured to estimate a failure probability of the ADS function over time in the virtual test environment by running a structural reliability method based on a parameterized statistical model and a Limit State Function (LSF), the LSF indicating a performance of the ADS function. In more detail, LSF g_i(θ) is the scene parameter set θ ═ θ₁,θ₂,…,θ_n]Indicates the operating environment of the ADS function. Also, LSF g_i(θ) includes:

a first function g_F(θ) which is a function of at least one scene parameter indicative of a fault scenario.

A second function g_S(θ) which is a function of at least one scenario parameter indicative of a severity level of the fault scenario, such that the estimated failure probability of the ADS function is further indicative of estimated failure probabilities for at least two different severity levels.

With this aspect of the disclosure, there are similar advantages and preferred functions as in the first aspect of the disclosure discussed above.

Further embodiments of the disclosure are defined in the dependent claims. It should be emphasized that the term "comprises/comprising" when used in this specification is taken to specify the presence of stated functions, integers, steps or components. This does not preclude the presence or addition of one or more other functions, integers, steps, components or groups thereof.

These and other features and advantages of the present disclosure will be further elucidated below with reference to the embodiments described hereinafter.

Drawings

Further objects, features and advantages of embodiments of the present disclosure will appear from the following detailed description, made with reference to the accompanying drawings, in which:

FIG. 1 is a schematic flow chart diagram representing a method for estimating failure probability for different severity levels of Automatic Driving System (ADS) functionality in a virtual test environment according to an embodiment of the present disclosure.

Fig. 2 is a schematic graph depicting an estimated failure probability of the ADS function generated by means of the method according to an embodiment of the present disclosure.

FIG. 3 is a schematic block diagram of a processing system representing a method for estimating failure probability for different severity levels of Automatic Driving System (ADS) functionality in a virtual test environment, according to an embodiment of the present disclosure.

Detailed Description

Those skilled in the art will appreciate that the steps, services and functions explained herein may be implemented using individual hardware circuitry, using software functioning in conjunction with a programmed microprocessor or general purpose computer, using one or more Application Specific Integrated Circuits (ASICs) and/or using one or more Digital Signal Processors (DSPs). It will also be appreciated that when the present disclosure is described in terms of methods, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, where the one or more memories store one or more programs that, when executed by the one or more processors, perform the steps, services, and functions disclosed herein.

In the following description of the exemplary embodiments, the same reference numerals denote the same or similar components.

As mentioned in the foregoing, it is not a simple task to evaluate whether a complex ADS function is compliant with the security specifications for different severity levels, and a brute-force approach may be said to prove to be infeasible. Moreover, performing a grid search or Monte-Carlo simulation may also be said to be ineffective due to the high dimensional parameter space of the statistical model and the low failure probability of the mature AD function with respect to the scene under test. Thus, the present inventors have recognized that structural reliability methods such as subset simulation (which may be abbreviated as SuS or SS) methods may be used in order to address this complex task. The structural reliability method can be understood as an advanced stochastic method for estimating the probability of a rare fault event, and was originally developed in the field of construction machinery.

Since they can efficiently transition scene simulation to a failure region (which is the case for ADS function simulation) in a high-dimensional parameter space, the present inventors have recognized that it can be advantageously used in software development and validation of ADS functions. Furthermore, the structural reliability method simulation allows to efficiently estimate the failure probability of the ADS function, given a simulation tool (software tool) and a multidimensional statistical model of the environment. Because the failure rate estimation is based on statistical models, which in turn are based on real traffic data, the failure rate estimation is highly relevant for the development and validation of ADS functions.

Structural reliability methods such as SuS use so-called limit-state functions (LSFs), which can be understood as continuous functions describing the performance of the system (i.e., the ADS function under test), to search for failure regions in the scene space. Thus, for an autonomous driving domain, the LSF may be based on, for example, Time To Collision (TTC) or time to post-intrusion (PET), both of which approach 0 when the situation approaches a collision.

In more detail, the subset simulation is based on the decomposition of rare events δ into more common events δ_iSo that the concept of the sequence of

Probability of occurrence of event delta

Very low, e.g. at 10^-9In the order of magnitude of (a) or (b),then directly estimate

It becomes very difficult, even sampling a single event requires a large amount of computational resources. However, using the subset simulation,

can be decomposed into delta for more frequent events_iEach of which evaluates a series of conditional probabilities,

wherein

Expressed at a given δ_i-1Event delta in case of occurrence_iThe conditional probability of (2). Also, in the SuS method, the conditional failure probability may be estimated by means of a markov chain monte carlo algorithm. In the above equation, the fault domain is represented by a set of values of a scenario parameter θ that renders unacceptable system performance:

δ＝{θ：g(θ)＜y^*} (3)

here, y ═ g (θ) represents the system performance LSF. As mentioned, the LSF may be a function of, for example, Time To Collision (TTC) or deviation from the center of the lane. In more detail, the LSF may be a function of the following variables:

1) the behavior of the ADS function under test. For example, it may be defined as the time of collision or the deviation from the center of the lane.

2) Scene parameter θ ═ θ describing an environment₁，θ₂，...，θ_n]Where the AD system performs, for example, target vehicle scene trajectories, a starting position of the target vehicle relative to the host machine, an initial state of the ego vehicle, an initial state of the ADs function, scene durations, and the like. Thus, "scene parameter θ describing an environment" should be interpreted broadly and is related not only to parameters describing an external state, but also to the performance of the ADS function thereinThe integrity case/scenario to be tested is relevant.

System performance y (e.g., TTC) and a specified threshold y^*(e.g., TTC ═ 0) defines the fault domain, i.e., if ≧ y^*Then the system can be inferred as "safe" and if y < y^*Then the system has failed. In other words, a fault domain is a set of parameters (scene trajectory, start position, duration, road curvature, etc.), where a simulation of the ADS function causes a collision (TTC ═ 0).

However, using only TTC or PET as the basis for LSF, we can effectively estimate the collision failure rate, but there is little nuance in the estimation and collisions of different severity levels cannot be distinguished. It should be noted that even though the discussion is primarily related to subset simulation (SuS), the skilled person readily recognizes how to adapt the concepts herein to other structural reliability approaches, and thus should not necessarily be construed as limiting, but merely as an example to clarify the concepts disclosed herein.

Thus, the present inventors recognized that we can provide an "extended" LSF, and then run a structural reliability method with an "extended" LSF, and thereby be able to estimate the failure probability for different severity levels. More specifically, extended LSF allows structural reliability methodology simulations to explore parameter spaces outside of the "fault case".

In more detail, an "extended" LSF as described herein may be understood as a piecewise function or a mixed function comprising two sub-functions, i.e. a first function and a second function as represented herein. The first function is a function of a parameter indicative of a fault scenario (e.g., a function of TTC), and the second function is a function of a scenario parameter indicative of a severity level of the fault scenario (e.g., a function of delta speed at collision). Thus, given that the second function is configured to obtain more negative values for more severe collisions, for example by using negative delta speeds at the time of a collision, it is possible to distinguish between different severity levels of the simulation results. Being able to generate failure probabilities for different severity levels may be advantageous to focus development and validation activities on the most needed regions/aspects of the system under test (ADS function under test).

In the illustrated example, assume that the "true" failure rate of ADS functionality in a particular scenario is 10 for an incident of severity level S1 (light or moderate injury)^-7Failure/hour and accident for severity level S3 (life threatening or fatal injury) is 10^-9Failure/hour. Further assume that the scene is a low controllability (C3) and a high exposure scene (E4).

Then, without extended LSF, conventional simulations may show that the failure probability of ADS function is 10 for low controllability (C3) and high exposure scenarios (E4)^-7Failure/hour. Now, without being able to differentiate between different severity levels, we will have to assume that it is the highest severity level (S3), which translates into an ASIL D requirement that can translate into an acceptable 10^-9Failure/hour failure rate. Thus, the simulation would indicate in this case that the ADS function is not performing adequately.

However, by using extended LSFs, as proposed herein, simulations may indicate a failure probability of 10^-7Failure/hour is for severity level S1, which translates to an acceptable failure rate of 10 therein^-7Satisfaction of failed/hour ASIL B requirements (S1, C3, E4). Moreover, the results may further indicate that the ADS function is at 10 for incidents of severity level S3^-9Failure rate per hour execution, which would indicate that the system actually meets ASIL D requirements. Thus, by allowing the structural reliability approach simulation to explore a parameter space beyond a simple "fault condition," more information can be obtained from the simulation, and validation/development activities can be properly focused where needed. It should be noted that even though the present disclosure pertains primarily to security objectives using the ISO26262 standard, this should not be construed as a limitation of the present disclosure, but merely serves to clarify and explain the teachings herein. Thus, the approach proposed herein may be used in conjunction with other known or presently unknown future standards related to "quantitative risk norms" in the automotive field.

Thus, according to embodiments of the method for estimating failure probability for different severity levels of ADS functions in a virtual test environment as disclosed herein, we present an effective tool for assessing compliance with security specifications using real-world statistics, which may drastically reduce the time and resources required for verification activities. Moreover, the results from the methods proposed herein can be used to generate important test cases for development, which can dramatically reduce the time and resources required for development activities.

FIG. 1 is a schematic flow chart diagram representing a method for estimating failure probability for different severity levels of ADS functionality in a virtual test environment. The ADS function is preferably, but not necessarily, an ADS function having a level 3 or higher according to SAE J3016 level of driving automation, such as a highway driver function, a traffic congestion driver function, and the like. Herein, the term "ADS function" or "autonomous driving function" may refer to any ADS, ADAS, or autonomous driving function, e.g., as already known in the art and/or as yet to be developed. The term "obtaining" is to be interpreted broadly herein and includes receiving, retrieving, collecting, obtaining, and the like.

The method 100 comprises obtaining (101) a parameterized statistical model 3 indicating a statistical distribution in an Operational Design Domain (ODD) of the ADS function to be tested with respect to a plurality of scenes in the real world environment. An Operational Design Domain (ODD) is to be understood as a description of the operating conditions in which an automated or semi-automated driving system (i.e., AD or ADAS) is designed to operate, including, but not limited to, geography, roads (e.g., type, ground, geometry, edges, and signs), environmental parameters, connectivity, surrounding objects, traffic parameters, and speed limits.

As a prerequisite, data 1 may be collected from real traffic scenes in order to gather statistical information of traffic scene dynamics (i.e. the probability of various situations occurring during driving). Further, a multivariate statistical model is fitted (106) to the aggregated parametric recording scene 1. The multivariate statistical model may for example be in the form of a gaussian mixture model. Then, a parameterized statistical model 3 of the scene of interest in a given ODD of the ADS function may be obtained (101).

With the term "statistical model", it can be understood as a description of what the ADS can expect statistically from its operating environment. In more detail, from the beginning, we can model live datasets by different segmentation and quantification methods (which may be referred to as "scene recognition"). In other words, the result of the scene recognition process is a set of scene parameters, and the statistical model is obtained by modeling the scene identified in the field data. Thus, a statistical model may be understood as a mathematical representation of a statistical distribution. In more detail, the statistical model of the environment statistically quantifies what the ADS can expect from its surroundings. In other words, the statistical model provides a probability measure for at least one scene that may occur within the environment (e.g., overtaking, pedestrian crossing, animal crossing, behavior of other road users, etc.). Thus, to statistically describe the environment, the statistical distributions corresponding to all scenes may be aggregated together to form a "global" statistical distribution. Further details regarding how statistical modeling and parameterized statistical models may be generated are disclosed, for example, in the present co-pending european patent application No. 20169897.4, by the same applicant, and incorporated herein by reference.

Further, the method 100 includes estimating (102) a failure probability of the ADS function over time in the virtual test environment by running (105) a structural reliability method, such as SuS, based on the parameterized statistical model and a constraint state function (LSF), wherein the LSF indicates performance of the ADS function. In more detail, LSF g_i(θ) is the scene parameter set θ ═ θ₁,θ₂,…,θ_n]Is used to "expand" the LSF in the form of a piecewise function of (a). LSF g_i(θ) includes a first function g_F(theta) and a second function g_S(θ), wherein F in the first function represents "fault" and S in the second function represents "severity". In more detail, the first function g_F(theta) is a function indicating at least one scene parameter of a fault scene, and a second function g_S(θ) is a function of at least one scenario parameter indicative of a severity level of the fault scenario. Thus, the estimated (102) failure probability of the ADS function is further indicative of an estimated failure probability 4 for at least two different severity levels.

In the context of the present disclosure, the statistical model may specifically indicate a scenario in the test for ADS functionality. For example, if the ADS function is tested against cut-ins, the statistical model includes information about the statistical distribution of the cut-ins. Thus, a statistical model for the estimation 102 of the failure probability of the ADS function may be obtained by fitting the model to all the plunges, or the statistical model may be extracted from a "global" statistical model that indicates a statistical distribution of multiple scenes of a complete ODD for the ADS function. Thus, to check for compliance with a set of predefined safety specifications for the ability of the ADS function to handle a cut-through, a statistical model indicating one or more statistical distributions about the cut-through is used as an input to the structural reliability method simulation. However, in some implementations, the statistical model for the estimation 102 of the failure probability of the ADS function may indicate a plurality of scenarios in the ODD of the ADS function. Thus, we can estimate the failure probability of the ADS function in its entire ODD, or at least for more than one specific scenario that the ADS is configured to handle.

Furthermore, the estimation 102 of the failure probability of the ADS function under test may comprise a sub-step, e.g. generating (iteratively) a parameter set θ from a statistical model_iSimulating (103) theta in a virtual test environment_iCorresponding scene, and evaluating (104) the ADS scene theta based on the output of the virtual simulation environment_iThe performance of (1). The iterative process is indicated by a feedback loop 105 associated with the subset simulation method. Specifically, SuS progressively explores the parameter space of the statistical simulation of the scenario under test to effectively produce the scenario in the failure region of the ADS function (as indicated in block 105'). This is done by automatically decomposing the failure region into a set of more frequent events, see equation (1). Therefore, probability of failure

Is decomposed into estimates of greater probability as shown in equation (2), where each conditional failure probability can be estimated by means of a markov chain monte carlo algorithm, e.g., a Modified Metropolis Algorithm (MMA). MMA is an advanced sampling technique suitable for producing samples from a conditional distribution. Specifically, the MMA performs mining by taking the following substepsSample preparation: generating (iteratively) a parameter set θ from a statistical model of a scene under test_iSimulating (103) theta in a virtual test environment_iCorresponding scene, and evaluating (104) the scene theta based on the output of the virtual simulation environment_iPerformance y of ADS function in_i＝g(θ_i). If y is_iIf the condition in equation (3) is satisfied, it is accepted, otherwise it is rejected and discarded. The MMA iteratively repeats (105) this process to produce enough samples in the fault domain (as indicated in block 7) of the ADS function under test to be able to reliably estimate the fault probability

Also, "outputs" from specific steps in the flow chart representing the method 100 are indicated in blocks 3, 7 and 105' in order to further clarify the concepts disclosed herein. As will be readily appreciated by the skilled reader, even though the illustrated embodiment of fig. 2 primarily represents subset simulation, the utilization of other structural reliability methods, such as significance sampling, are equally applicable. However, for the sake of brevity and conciseness, these alternative embodiments will not be depicted in the drawings.

The effect of the simulation according to embodiments disclosed herein is illustrated in fig. 2, fig. 2 showing a graph of the fault estimation probability with respect to Time To Collision (TTC). In particular, the effect provided by the embodiments disclosed herein is indicated in the dashed box 20. In more detail, a possibility of continuing exploration beyond just "collision" (i.e., the event at TTC ═ 0) to "collision of different severity" is provided. The effect of "expanding" the LSF can be seen in the dashed portion 22 of the

straight line graphs

21, 22, which dashed portion 22 continues beyond the "fault" scenario (i.e. TTC 0), whereby we can obtain estimates of the probability of fault for different severity levels S0, S1, S2 and S3.

From the simulation, we can then extract the estimated

results

23, 24 and compare them with, for example, safety specifications indicating maximum failure rates for different severity levels, as indicated below in table 1.

Table 1: an example of evaluating compliance with safety regulations (ASIL) based on the results from fig. 2, assume that the exposure level (E) is E4 and the controllability level (C) is C3.

It should be noted that the ISO standard does not assign any ASIL requirement to the event that is S0, and the event is referred to as Quality Management (QM). This means that ISO does not suggest any upper limit on the failure rate of QM and leaves this to companies to design products that can meet the needs of consumers. Thus, the company designing ADS sets an upper limit on the failure rate of the S0 event. However, the method proposed herein can still be used to check whether the performance of the ADS function satisfies such conditions.

Returning to the discussion regarding the extended constraint State function (LSF), for a given ADS function, it can now be computed that the encounter has a severity level S_iProbability Pr (S) of scene (i ∈ 0, 1, 2, 3)_i). In more detail, the inventors realized that severity is only assigned to the scenario that caused the failure (collision), which is equal to Pr (S)_iδ), i.e., encounter with severity level S_iThe probability of collision of (2). Using the definition of conditional probabilities, this can be rewritten as:

Pr(S_i，δ)＝Pr(δ)·Pr(S_i|δ) (4)

the probability in equation (4) can be approximately calculated by means of a structural reliability method such as SuS. For this purpose, the LSF may be defined as:

g_i(θ)＝g_F(θ)-g_S(θ)+c_i (5)

the LSF in equation (5) thus includes three "parts", i.e., the first function g_F(theta), second function g_S(theta) and a level parameter c defining a threshold for the severity level of the fault scenario_i。

According to some embodiments, the first function

Is a continuous function, which is defined such that g_F(θ) 0 corresponds to collision and g_F(θ) ≧ 0 corresponds to a non-collision event. Typical measures used in the automotive field, which easily meet this, are, for example, the Time To Collision (TTC) and the time to post-intrusion (PET). However, other threat metrics such as the number of Braking Threats (BTN) that monotonically increase may be used by using an appropriate transformation such that the above condition is satisfied. Another example threat metric that may be used is Time To Brake (TTB). In other words, according to some embodiments, the first function g_F(θ) is defined as having a value of zero for a fault scenario and a non-zero value for a non-fault scenario.

According to some embodiments, the second function

Is a continuous function which represents the severity of the collision, such that g_SHigher values of (θ) correspond to more severe collisions. For a non-collision scenario, the second function may be defined as g_S(θ) ═ 0. In some embodiments, the absolute incremental velocity | Δ ν during a collision may be used_col(theta) | to get g_S(θ) is defined as:

thus, in some embodiments, the second function g_S(θ) is defined as having a value of zero for non-failure scenarios and a non-zero value for failure scenarios. Also, according to some embodiments, the second function g_S(θ) is a function of at least one of an incremental speed at the time of the collision, a weight of the vehicle, a weight of the colliding object, an absolute speed of the vehicle at the time of the collision, an angle of the vehicle at the time of the collision, a restitution factor, and a point of impact at the time of the collision. In other words, the second function gS (θ) may be a function of at least one variable indicating the severity of a fault scenario (e.g., collision, departure from a road, etc.). Further, the second function may be a function of a sub-function indicative of the severity of injury to the occupant, which may be the age of the occupant, the cockpit safety beltThe use of (c), etc. The sub-function (which may also be referred to as an injury severity risk function) may be estimated based on historical data from an accident database.

Also, a threshold value c may be used_i(i ∈ 0.., n +1) is based on g_S(theta) to define a severity level S_iI.e. if and only if c_i＜g_S(θ_j)≤c_i+1Time of flight collision scenario θ_jE δ is considered to have a severity level S_i. Severity level S₀Lower threshold value c ₁0 and severity level S_nUpper threshold value c of_n+1Infinity. According to the level of the interesting scene, the corresponding threshold value c_iIs used as the constant c in equation (5)_i. In this way, corresponding to, for example, having a severity level S₃The domain of failure of a collision of (a) can be broken down into more frequent events:

this means that in a single run SuS continuously forms more frequent domains to be able to move towards the desired fault domain, i.e. the collision scenario

And (5) advancing.

Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

FIG. 3 is a schematic block diagram representing a processing system 10 for estimating failure probability for different severity levels of ADS functionality in a virtual test environment. The processing system 10 includes control circuitry 11 (which may also be referred to as a control unit, controller, processor (s)), memory 12, a communication interface 13, and any other conventional components/functions required for performing a method according to any of the embodiments disclosed herein. In other words, executable instructions 14 for performing these functions are optionally included in a non-transitory computer-readable storage medium 12 or other computer program product configured for execution by one or more processors 11.

In more detail, the control circuit 11 is configured to obtain a parameterized statistical model indicating a statistical distribution in an Operational Design Domain (ODD) of the ADS function with respect to a plurality of scenes in the real-world environment. Further, the control circuit 11 is configured to estimate a failure probability of the ADS function over time in the virtual test environment by running a structural reliability method based on a parameterized statistical model and a Limit State Function (LSF), the LSF indicating a performance of the ADS function. In more detail, LSF g_i(θ) is a scene parameter set θ ═ θ indicating the operating environment of the ADS function₁,θ₂,…,θ_n]As a function of (c). Also, LSF g_i(θ) includes:

In summary, the solution proposed herein enables the estimation of the probability of collisions of different severity by making use of a Limit State Function (LSF) which obtains increasingly negative or increasingly positive values after a collision (e.g. when TTC is 0 or PET is 0). This may be accomplished, for example, by defining a function that is more negative for more severe crash severity. According to an example embodiment, the LSF includes a function of the delta speed at impact (i.e., a negative delta speed at impact). In this way, higher incremental speeds at impact achieve more negative values, which translates into a higher severity of impact. For example, there are different studies showing how to map severity with delta speed (e.g., S1 for Δ V >35kph, S2 for Δ V >70, etc.).

However, it may be desirable to define a more detailed severity function (i.e., a sub-function of the LSF) that further considers, for example, the absolute speed at impact, the angle and point of impact, the impacting vehicle characteristics (weight, height, features), and the like. Moreover, with the concepts proposed herein, severity can also be estimated for the host and other vehicles/participants. This may be advantageous, for example, for scenarios involving motorcycles in which the speed for the same severity level is typically lower.

The present disclosure has been presented above with reference to specific embodiments. However, other embodiments than the above described are possible and are within the scope of the present disclosure. Different method steps than those described above, performing the method by hardware or software, may be provided within the scope of the present disclosure. Thus, according to an exemplary embodiment, a non-transitory computer readable storage medium is provided storing one or more programs configured for execution by one or more processors of a vehicle control system, the one or more programs including instructions for performing a method according to any of the embodiments discussed above. Alternatively, according to another exemplary embodiment, the cloud computing system may be configured to perform any of the methods presented herein. The cloud computing system may comprise distributed cloud computing resources that jointly execute the methods presented herein under the control of one or more computer program products.

Generally speaking, a computer-accessible medium may include any tangible or non-transitory storage or memory medium, such as an electronic, magnetic, or optical medium, such as a diskette or CD/DVD-ROM coupled to a computer system via a bus. The terms "tangible" and "non-transitory" as used herein are meant to describe computer-readable storage media (or "memory") that do not include propagated electromagnetic signals, but are not meant to otherwise limit the type of physical computer-readable storage device encompassed by the phrase "computer-readable media or memory. For example, the terms "non-transitory computer readable medium" or "tangible memory" are intended to encompass types of storage devices that do not necessarily permanently store information, including, for example, Random Access Memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer accessible storage medium may further be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be transmitted over communication media such as a network and/or a wireless link.

The processor 11 (associated with the control system 10) may be or include any number of hardware components for performing data or signal processing or for executing computer code 14 stored in memory 12. The device 10 has associated memory 12, and the memory 12 may be one or more devices for storing data and/or computer code for performing or facilitating the various methods described in this specification. The memory may include volatile memory or non-volatile memory. Memory 12 may include database components, object code components, script components, or any other type of information structure for supporting the various activities of the specification. According to exemplary embodiments, any distributed or local memory device may be used with the systems and methods of this specification. According to an exemplary embodiment, memory 12 is communicatively connected to processor 11 (e.g., via circuitry or any other wired, wireless, or network connection) and includes computer code for performing one or more of the processes described herein.

It should be noted that the word "comprising" does not exclude the presence of other elements or steps than those listed and the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. It should further be noted that any reference signs do not limit the scope of the claims, that the disclosure may be at least partially implemented by means of both hardware and software, and that several "means" or "units" may be represented by the same item of hardware. It will also be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first function may be referred to as a second function, and similarly, a second function may also be referred to as a first function, without departing from the scope of the embodiments. The first function and the second function are both functions, but they are not the same function.

Although the figures may show a particular order of method steps, the order of the steps may differ from that which is described. Further, two or more steps may be performed simultaneously or partially simultaneously. Such variations will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the present disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. The embodiments mentioned and described above are given by way of example only and should not be limiting to the present disclosure. Other aspects, uses, objects, and functions within the scope of the disclosure as claimed should be apparent to one skilled in the art.

Claims

1. A computer-implemented method for estimating failure probability for different severity levels of an autonomous driving system, ADS, function in a virtual test environment, the method comprising:

obtaining a parameterized statistical model indicating statistical distribution in an operating design domain, ODD, of the ADS function with respect to at least one scene in a real-world environment;

estimating the fault probability of the ADS function in the virtual test environment along with time by operating a structural reliability method based on the parameterized statistical model and the LSF, wherein the LSF indicates the performance of the ADS function;

wherein the LSF g_i(theta) is a scene parameter set theta [ theta ] indicating an operation environment of the ADS function₁,θ₂,…,θ_n]The LSF g is_i(θ) includes:

first function g_F(θ) which is a function of at least one scene parameter indicative of a fault scene;

second function g_S(θ) which is a function of at least one scenario parameter indicating a severity level of a fault scenario, such that the estimated probability of failure of the ADS function is further indicative of estimated probability of failure for at least two different severity levels.

2. The meter of claim 1A computer-implemented method, wherein said first function g_F(θ) is defined as having a value of zero for a fault scenario and a non-zero value for a non-fault scenario; and

wherein the second function g_S(θ) is defined as having a value of zero for non-failure scenarios and a non-zero value for failure scenarios.

3. The computer-implemented method of claim 1, wherein the structural reliability method is a subset simulation method.

4. The computer-implemented method of any of claims 1-3, wherein the LSF g_i(θ) further comprises a level parameter c defining a threshold for a severity level of the fault scenario_i。

5. The computer-implemented method of any of claims 1-3, wherein the first function g_F(θ) is a function of at least one of time to collision TTC, time to post-intrusion PET, and number of braking threats BTN; and

wherein the second function g_S(θ) is a function of at least one of an incremental speed at the time of the collision, a weight of the vehicle, a weight of the colliding object, an absolute speed of the vehicle at the time of the collision, an angle of the vehicle at the time of the collision, a restitution factor, and a point of impact at the time of the collision.

6. A computer readable storage medium storing one or more programs configured for execution by one or more processors of a processing system, the one or more programs comprising instructions for performing the method of any of the preceding claims.

7. A processing system for estimating failure probability for different severity levels of an automatic driving system, ADS, function in a virtual test environment, the processing system comprising:

a control circuit configured to:

second function g_S(θ) which is a function of at least one scenario parameter indicative of a severity level of the fault scenario, such that the estimated probability of failure of the ADS function is further indicative of estimated probability of failure for at least two different severity levels.

8. The processing system of claim 7, wherein the first function g_F(θ) is defined as having a value of zero for a fault scenario and a non-zero value for a non-fault scenario; and

wherein the second function g_S(θ) is defined as having a value of zero for non-failure scenarios and a value of non-zero for failure scenarios.

9. The processing system of claim 7, wherein the structural reliability method is a subset simulation method.

10. The processing system of any of claims 7-9, wherein the first function g_F(θ) is a function of at least one of time to collision TTC, time to post-intrusion PET, and number of braking threats BTN; and

wherein the second function g_S(theta) is the incremental speed at the time of collision, vehicleA weight of the impacting object, an absolute velocity of the vehicle at the time of the impact, an angle of the vehicle at the time of the impact, a recovery factor, and a point of impact at the time of the impact.