CN114428998A - Integrated simulation test and evaluation method and system for automatic driving system - Google Patents
Integrated simulation test and evaluation method and system for automatic driving system Download PDFInfo
- Publication number
- CN114428998A CN114428998A CN202210100682.8A CN202210100682A CN114428998A CN 114428998 A CN114428998 A CN 114428998A CN 202210100682 A CN202210100682 A CN 202210100682A CN 114428998 A CN114428998 A CN 114428998A
- Authority
- CN
- China
- Prior art keywords
- test
- library
- simulation
- traffic flow
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/15—Vehicle, aircraft or watercraft design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/14—Force analysis or force optimisation, e.g. static or dynamic forces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Evolutionary Computation (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of automatic driving system testing, in particular to an integrated simulation testing and evaluating method and system of an automatic driving system. The invention integrates the automatic driving system test based on the scene and the automatic driving system test based on the traffic flow, fully exerts the advantages of the two test methods, improves the test efficiency, evaluates the automatic driving system from two aspects of the safety of the vehicle and the influence on the traffic safety, has more comprehensive evaluation angle and more accurate evaluation result.
Description
Technical Field
The invention relates to the technical field of automatic driving system testing, in particular to an integrated simulation testing and evaluating method and system for an automatic driving system.
Background
The traffic accident brings serious injury to human beings, a driver is used as the strongest random factor in the traffic environment and is also the main cause of the traffic accident, and the automatic driving of the automobile is acknowledged as an effective way for avoiding or reducing the traffic accident caused by human errors aiming at the problem of the traffic accident. The automatic driving system is software for realizing a driving control function on the automatic driving automobile, and in order to ensure the accuracy and the safety of the driving of the automatic driving automobile, the automatic driving system on the automatic driving automobile needs to be tested and evaluated before the automatic driving automobile drives on the road. In particular, in a high-grade autonomous driving automobile, i.e., an autonomous driving automobile of SAE L3 grade or above, a driver does not always have the right to control the motion of the vehicle during operation, and an autonomous driving system becomes a main body for monitoring the driving environment and controlling the operation of the vehicle.
The conventional simulation test method for the automatic driving system is mainly based on standard laws and regulations, a scene segment is designed in advance, and then the safety of the automatic driving system is evaluated according to the passing ability. However, because the randomness of the traffic environment and the interactivity among traffic participants are ignored in the test method, the test method greatly differs from the real environment when the vehicle runs, the coverage requirement of a test scene is difficult to meet, and the existing method cannot test and evaluate the influence of the automatic driving system on the traffic environment, such as traffic passing efficiency, traffic conflict safety and the like, so that the evaluation result of the automatic driving system is not accurate enough.
Disclosure of Invention
The invention aims to provide an integrated simulation test and evaluation method of an automatic driving system, so as to improve the accuracy of test evaluation of the automatic driving system.
The integrated simulation test and evaluation method of the automatic driving system in the scheme comprises the following steps:
step 1, building a vehicle dynamic model of an automatic driving vehicle and storing the vehicle dynamic model into a dynamic model library, building a scene model and storing the scene model into a test scene library, building a test case and storing the test case into a test case library, building a traffic flow model and storing the traffic flow model into a traffic flow library;
further comprising the steps of:
step 2, establishing a mapping relation between the traffic flow model and the scene model according to preset map information in the traffic flow model and the scene model;
step 3, acquiring test requirements, establishing a simulation calculation center for simulation test, and scheduling the dynamic model library, the test scene library, the test case library and the traffic flow library according to the acquired test requirements during simulation;
step 4, after dispatching, simulating according to simulation configuration data in the test requirement, storing the simulation data, and collecting a simulation result in the simulation process;
and 5, performing simulation evaluation on the automatic driving system according to the used test scene library or traffic flow library, and generating an evaluation report.
The beneficial effect of this scheme is:
the test scene, the test case and the traffic flow are added in the simulation process, the specific interactive scene of the automatic driving vehicle in the actual driving process is added in the simulation, the automatic driving system test based on the scene and the automatic driving system test based on the traffic flow are integrated, the advantages of the two test methods are fully exerted, the test efficiency is improved, the simulation result is more accurate, meanwhile, the automatic driving system is evaluated from two angles of the safety of the vehicle and the influence on the traffic safety, the evaluation angle is more comprehensive, and the evaluation result is more accurate.
Further, in the step 1, a test case is formed by combining the test scene element, the complexity of the test scene and the passing condition of the test scene, wherein the test case library includes an ODD-based test case sub library, a functional safety-based test case sub library, an expected functional safety-based test case sub library and an expert experience-based test case sub library, the complexity is a weighted value of the scene element complexity and the driving task complexity, the scene element complexity is quantized according to the types of the static element and the dynamic element in the test scene, and the driving task complexity is quantized according to the type of the driving task in the test scene in unit time.
The beneficial effects are that: test cases are formed from multiple aspects, and test cases under different conditions are formed, so that a test scene is more complete and closer to an actual driving condition, and the accuracy of simulation test is improved.
Further, in the step 4, when the test scene library is used, the test functions, the test targets, the number of the test scenes, and the distribution characteristics of the test scenes are used as simulation data, and when the traffic flow library is used, the map file, the traffic flow density, the vehicle distribution, and the driver style distribution are used as simulation data.
The beneficial effects are that: when different libraries are used for simulation, different data are used as simulation data, and evaluation results are more accurate when evaluation is performed in different aspects.
Further, in step 5, when the test scenario library is used for simulation, the complexity, coverage and passability of the test case are graded according to a first model, where the first model is:
the coverage of the test case is as follows:
wherein: m isoddTotal number of test cases, m, for ODD-based test case sub-libraryfTotal number of test cases, m, for a library of test case sub-libraries based on functional securitysTotal number of test cases, m, for a library of test cases based on expected functional securityeThe total number of test cases of the test case sub-library based on expert experience; n isoddTotal number of real test cases, n, from the ODD-based test case sub-library among the test cases for the actual testfThe total number n of actual test cases from the function safety-based test case sub-library in the test cases for actual testsThe total number of real test cases, n, from the test case sub-library based on the expected functional safety in the test cases for actual testeThe total number of real test cases derived from the expert experience-based test case sub-library in the test cases for actual testing.
The beneficial effects are that: the safety grade of the automatic driving system is graded through the first model, quantitative evaluation is carried out from the safety of the automatic driving vehicle, and the accuracy of vehicle safety evaluation is improved.
Further, in the step 5, when the simulation is performed by using the traffic flow library, the simulation evaluation is performed on the traffic conflict of the automatic driving system by using the state information, the trajectory information and the traffic flow density information of the traffic participants.
The beneficial effects are that: from the perspective of traffic of actual running of the vehicle, the safety of the automatic driving system is evaluated, the integrity of the evaluation angle is improved, and the safety evaluation of the vehicle is more accurate.
Further, in the step 5, the traffic conflict is represented by the alternative variables of headway, inter-vehicle distance and collision time, when there is a traffic conflict, the occurrence frequency of the traffic conflict is collected to perform simulation evaluation, the occurrence frequency is the ratio of the number of conflicts to the number of miles driven, the position and the moment when the traffic conflict occurs are extracted, and the time distribution of the traffic conflict and the space distribution of the traffic conflict are formed according to the position and the moment.
The beneficial effects are that: and the traffic conflict is acquired through the corresponding parameters, and simulation evaluation is performed according to the traffic conflict, so that the accuracy and comprehensiveness of the simulation evaluation are improved.
And further comprising a step 6 of carrying out weighted comprehensive evaluation on simulation results under the two conditions when the test scene library and the traffic flow library are used for simulation at the same time to obtain a comprehensive performance safety evaluation value of the safety of the automatic driving automobile.
The beneficial effects are that: the simulation of the scene and the traffic flow at two angles is comprehensively evaluated, so that the overall evaluation of the automatic driving system is more reliable.
The integrated simulation test and evaluation system of the automatic driving system comprises a simulation interaction module, a modeling module, a database module, a processing module, a scheduling module, a sensor module and a calculation module;
the simulation interaction module is used for acquiring a test requirement and sending the test requirement to the processing module;
the system comprises a modeling module, a traffic flow model and a traffic flow model, wherein the modeling module is used for establishing a vehicle dynamics model of an automatic driving vehicle, the modeling module is used for establishing a scene model, the modeling module is used for establishing a test case, and the modeling module is used for establishing a traffic flow model;
the database module comprises a dynamic model library, a test scene library, a test case library and a traffic flow library which are used for storing the dynamic model library, the test scene library, the test case library and the traffic flow library which are required by the test;
the system comprises a processing module, a database module, a traffic flow model acquisition module, a data processing module and a data processing module, wherein the processing module is used for acquiring a vehicle dynamic model and storing the vehicle dynamic model into a dynamic model library in the database module, the processing module is used for acquiring a scene model and storing the scene model into a test scene library in the database module, the processing module is used for acquiring a test case and storing the test case into a test case library in the database module, and the processing module is used for acquiring a traffic flow model and storing the traffic flow model into a traffic flow library in the database module;
the dispatching module is used for dispatching the dynamic model library, the test scene library, the test case library and the traffic flow library according to the test requirements;
the sensor module is used for detecting a simulation result in a simulation test process;
and the calculation module is used for receiving the simulation result acquired by the processing module from the sensor module, calculating simulation evaluation according to the simulation result and generating an evaluation report.
Drawings
Fig. 1 is a schematic block diagram of an integrated simulation test and evaluation system of an automatic driving system according to a first embodiment of the present invention;
FIG. 2 is a block diagram of a flow chart of an integrated simulation testing and evaluating method of an automatic driving system according to a second embodiment of the present invention;
FIG. 3 is a schematic block diagram illustrating the complexity calculation of a test scenario in the integrated simulation test and evaluation method of the automatic driving system according to the second embodiment of the present invention;
FIG. 4 is a schematic block diagram of test case generation in the integrated simulation test and evaluation method of the automatic driving system according to the second embodiment of the present invention;
fig. 5 is a schematic diagram of a data mapping relationship between models of a computing center in the integrated simulation test and evaluation method of the automatic driving system in the second embodiment of the present invention;
fig. 6 is a schematic view of a test execution and evaluation flow of simulation under a scene model in the integrated simulation test and evaluation method of the automatic driving system in the second embodiment of the present invention;
fig. 7 is a schematic view of a test execution and evaluation flow of simulation under a traffic flow model in the integrated simulation test and evaluation method of the automatic driving system in the second embodiment of the present invention.
Detailed Description
The following is a more detailed description of the present invention by way of specific embodiments.
Example one
An integrated simulation test and evaluation system of an automatic driving system is shown in fig. 1: the system comprises a simulation interaction module, a modeling module, a database module, a processing module, a scheduling module, a sensor module and a calculation module.
The simulation interaction module is used for acquiring a test requirement and sending the test requirement to the processing module, the test requirement is expressed by simulation configuration parameters required to be set in the test process, the simulation interaction module can acquire the test requirement through the existing input peripheral devices such as a keyboard and a touch screen, and the existing display screen is used for displaying the simulation configuration parameters required to be set.
The modeling module is used for establishing a vehicle dynamics model of the automatic driving vehicle, the modeling module is used for establishing a scene model, the modeling module is used for establishing a test case, the modeling module is used for establishing a traffic flow model, the modeling module can establish various models through the existing software carried on a PC host or a notebook computer, and the existing modeling mode and the software carried on the existing modeling module are used for establishing various models.
The database module can be built by using the existing database software, such as Oracle software, and comprises a dynamic model library, a test scene library, a test case library and a traffic flow library which are used for storing and testing.
The processing module is used for acquiring vehicle dynamic models and storing the vehicle dynamic models into a dynamic model library in the database module, the processing module is used for acquiring scene models and storing the scene models into a test scene library in the database module, the processing module is used for acquiring test cases and storing the test cases into a test case library in the database module, and the processing module is used for acquiring traffic flow models and storing the traffic flow models into a traffic flow library in the database module; the processing module establishes a mapping relation between the traffic flow model and the scene model according to the map information, the map information is generated when the traffic flow model and the scene model are established, for example, the processing module establishes a mapping relation between a scene model file and a traffic flow model file with the same static map in a mode of same file name prefix.
The scheduling module is used for scheduling the dynamic model library, the test scene library, the test case library and the traffic flow library according to the test requirements, for example, scheduling the model library according to whether the test requirements are scene-based tests, traffic flow-based tests or both the scene and the traffic flow.
The sensor module is used for detecting simulation results in a simulation test process, the simulation results comprise corresponding parameters obtained through simulation under test scenes and traffic flow conditions, and the sensor module can be used for sensors required in an existing automatic driving system, such as a distance measuring sensor, an image sensor and the like.
The calculation module is configured to receive the simulation result obtained by the processing module from the sensor module, calculate a simulation evaluation according to the simulation result, generate an evaluation report, and calculate the simulation evaluation according to a preset calculation formula according to parameters in the obtained simulation result, as in the formula of the method in embodiment two, where the evaluation report includes scores or graphs of simulation tests of the vehicle dynamics model under different conditions.
According to the system, the vehicle is modeled through the modeling module, simulation tests of a test scene and traffic flow are carried out on the modeled vehicle model under different test cases, the vehicle model is automatically scheduled after being mapped in the simulation process, simulation tests are carried out on the running states of the automatic driving vehicle under different conditions through different library mappings, the test conditions are closer to the actual driving conditions, the test simulation is closer to the actual running environment, the accuracy of the test simulation is improved, and the problems of the automatic driving system can be found in advance.
Example two
An integrated simulation test and evaluation method of an automatic driving system, which utilizes the integrated simulation test and evaluation system of the automatic driving system of the first embodiment, as shown in fig. 2, includes the following steps:
step 1, a vehicle dynamics model of the automatic driving vehicle is established through a modeling module and stored in a dynamics model library, the vehicle dynamics model of the automatic driving vehicle is established according to an actual vehicle to be tested, the vehicle dynamics model is established by using the existing vehicle dynamics software and the existing technology, and details are not repeated here.
The method comprises the steps of establishing a scene model and storing the scene model into a test scene library, respectively establishing the scene model based on a design operation domain (ODD), functional safety, expected functional safety and expert experience specified by the existing standard of an automatic driving system, wherein the stored format of the scene model is an open-x format, the scene model is established by using the existing VTD software, such as the scene model of a city expressway exit scene, and the test scene library comprises a scene sub-library based on the ODD, a scene sub-library based on the functional safety, a scene sub-library based on the preset functional safety and a scene sub-library based on the expert experience.
As shown in fig. 4, creating and storing test cases into a test case library, and combining and integrating test scenario elements, complexity of the test scenario and passing conditions of the test scenario into a test case, where the test case library includes an ODD-based test case sub-library, a functional safety-based test case sub-library, a test case sub-library based on expected functional safety and an expert experience-based test case sub-library, as shown in fig. 3, the complexity is a scene element complexity and driving task complexity weighted value, such as a weighted value of a scene element complexity a multiplied by a weight w1 and a driving task complexity multiplied by a weight w2, the weights w1 and w2 are set according to simulation test requirements, such as w1 4 and w2, the scene element complexity is quantified according to static element and dynamic element composition types in the test scenario, such as a sum of composition types of the static element and a dynamic element composition type of the static element and a sum of the dynamic element composition types in the test scenario, the driving task complexity is quantified according to the driving task types in the test scene in unit time, for example, the number P of the driving task types in T unit time is taken as the driving task complexity.
And establishing a traffic flow model and storing the traffic flow model into a traffic flow library, wherein the establishment of the traffic flow model is respectively established based on the design operation domain, the functional safety, the expected functional safety and the expert experience of an automatic driving system, the traffic flow model can be established by using the conventional VISSIM software, and the traffic flow model is the average traffic quantity Q, the average speed V, the average density K, the track information of traffic participants and the state information of the traffic participants on a certain route. And storing the model and the test case into a database module after the model and the test case are established.
Step 2, the map information is uniformly preset when the traffic flow model and the scene model are established through the processing module according to the preset map information in the traffic flow model and the scene model, as shown in fig. 5, a mapping relation is established between the traffic flow model and the scene model in the test scene library, for example, a mapping relation is established between a scene model file with the same static map and a traffic flow model file with the same file name prefix.
And 3, acquiring a test requirement through the simulation interaction module, establishing a simulation computing center for simulation test, scheduling the dynamic model library, the test scene library, the test example library and the traffic flow library according to the acquired test requirement through the scheduling module during simulation, namely scheduling the model library in the simulation software according to the condition that the test requirement is based on scene test, traffic flow test or both scene and traffic flow, for example, when the test requirement is based on scene test for an A-type automatic driving vehicle, calling a vehicle dynamic model with a corresponding model from the dynamic model library, starting the vehicle dynamic software and the scene simulation software, and establishing API (application program interface) communication connection of the vehicle dynamic software and the scene simulation software.
And 4, after scheduling, simulating by the processing module according to the simulation configuration data in the test requirement, storing the simulation data, and acquiring a simulation result in the simulation process, wherein the simulation result comprises corresponding parameters obtained by simulation under the conditions of a test scene and a traffic flow. When the test scene library is used, the test function, the test target, the number of the test scenes and the distribution characteristics of the test scenes are used as simulation data, when the traffic flow library is used, the map file, the traffic flow density, the vehicle distribution and the driver style distribution are used as the simulation data, the traffic flow density is the number of vehicles passing through in unit time, the vehicle distribution is the proportion of a trolley, a truck and a two-wheeled vehicle in the total amount, the driver style distribution comprises an aggressive type, a normal type and a conservative type, and the map file is map information in a traffic flow model and a scene model.
Step 5, the calculation module performs simulation evaluation on the automatic driving system according to the used test scene library or traffic flow library, and generates an evaluation report, as shown in fig. 6, when the test scene library is used for simulation, the complexity, coverage and trafficability of the test case are graded according to a first model, wherein the first model is as follows:
the coverage of the test case is as follows:
wherein: m isoddTotal number of test cases, m, for ODD-based test case sub-libraryfTotal number of test cases, m, for a library of test case sub-libraries based on functional securitysTotal number of test cases, m, for a library of test cases based on expected functional securityeThe total number of test cases of the test case sub-library based on expert experience; n is a radical of an alkyl radicaloddTotal number of real test cases, n, from the ODD-based test case sub-library among the test cases for the actual testfThe total number of real test cases, n, from the function safety-based test case sub-library in the test cases for actual testsThe total number of real test cases, n, from the test case sub-library based on the expected functional safety in the test cases for actual testeThe total number of real test cases, n, derived from the expert experience-based test case sub-library in the test cases for actual testingodd、nf、nsAnd neFrom simulated configuration parameters in test requirements.
As shown in fig. 7, when a traffic flow library is used for simulation, the state information, the track information and the traffic flow density information of traffic participants are used for simulation evaluation of the traffic conflict of the automatic driving system, so as to replace variables of headway, distance between vehicles and collision time to represent the traffic conflict, when there is a traffic conflict, the occurrence frequency of the traffic conflict is collected for simulation evaluation, the occurrence frequency is the ratio (CR) of the number of conflicts to the mileage of travel, the position and the moment when the traffic conflict occurs are extracted, the time distribution of the traffic conflict and the space distribution of the traffic conflict are formed by the position and the moment, and the time distribution is a two-dimensional diagram: the abscissa of the time coordinate is a time period in one day, and the ordinate is the collision frequency; the spatial distribution is a three-dimensional map: and the horizontal and vertical coordinates are respectively global geographic position coordinates, and the traffic conflict frequency is displayed in the form of thermodynamic diagrams.
Step 6, when the test scene library and the traffic flow library are used for simulation at the same time, carrying out weighted comprehensive evaluation on simulation results under two conditions to obtain a comprehensive performance safety score value of the safety of the automatic driving automobile, which is expressed as:
Safetyall＝CR×K1+Safety×K2the method comprises the steps of obtaining a ratio of a conflict number to a travel mileage number according to a traffic flow simulation test, obtaining a specific value of a weighted value K1, obtaining a specific value K1 according to an actual requirement, obtaining a Safety level number according to a scene simulation test, obtaining a corresponding weighted value K2, and obtaining a specific value K2 according to the actual requirement.
Because the automatic driving automobile can avoid man-made subjective negligence or error under the conventional thinking, the automatic driving automobile is considered to have high safety factor, so that when the evaluation of the automatic driving system is carried out, the requirement of corresponding standard is generally met according to the automatic driving system, namely the safety of the automatic driving system is considered to meet the requirement. In the method of the embodiment, the test case, the test scene and the traffic flow are set for the performance simulation evaluation of the automatic driving system on the automatic driving automobile, the influence of the environment and the traffic flow in actual driving can be considered, the test can be carried out from multiple dimensions, the evaluation is carried out according to the test result, and the test accuracy and the evaluation accuracy are improved.
The foregoing is merely an example of the present invention and common general knowledge of known specific structures and features of the embodiments is not described herein in any greater detail. It should be noted that, for those skilled in the art, without departing from the structure of the present invention, several changes and modifications can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicability of the patent. The scope of the claims of the present application shall be determined by the contents of the claims, and the description of the embodiments and the like in the specification shall be used to explain the contents of the claims.
Claims (8)
1. An integrated simulation test and evaluation method for an automatic driving system comprises the following steps:
step 1, building a vehicle dynamic model of an automatic driving vehicle and storing the vehicle dynamic model into a dynamic model library, building a scene model and storing the scene model into a test scene library, building a test case and storing the test case into a test case library, building a traffic flow model and storing the traffic flow model into a traffic flow library; the method is characterized in that: further comprising the steps of:
step 2, establishing a mapping relation between the traffic flow model and the scene model according to preset map information in the traffic flow model and the scene model;
step 3, acquiring test requirements, establishing a simulation calculation center for simulation test, and scheduling the dynamic model library, the test scene library, the test case library and the traffic flow library according to the acquired test requirements during simulation;
step 4, after dispatching, simulating according to simulation configuration data in the test requirement, storing simulation data, and collecting a simulation result in the simulation process;
and 5, performing simulation evaluation on the automatic driving system according to the used test scene library or traffic flow library, and generating an evaluation report.
2. The integrated simulation test and evaluation method of the automatic driving system according to claim 1, characterized in that: in the step 1, the test scenario elements, the complexity of the test scenario and the passing condition of the test scenario are combined to form a test case, the test case library comprises an ODD-based test case sub-library, a functional safety-based test case sub-library, an expected functional safety-based test case sub-library and an expert experience-based test case sub-library, the complexity is a weighted value of the complexity of the scenario elements and the complexity of the driving task, the complexity of the scenario elements is quantized according to the types of the components of the static elements and the dynamic elements in the test scenario, and the complexity of the driving task is quantized according to the types of the driving task in the test scenario within unit time.
3. The integrated simulation test and evaluation method of the automatic driving system according to claim 2, characterized in that: in the step 4, when the test scene library is used, the test functions, the test targets, the number of the test scenes and the distribution characteristics of the test scenes are taken as simulation data, and when the traffic flow library is used, the map file, the traffic flow density, the vehicle distribution and the driver style distribution are taken as simulation data.
4. The integrated simulation test and evaluation method of the automatic driving system according to claim 3, characterized in that: in the step 5, when the test scene library is used for simulation, the complexity, the coverage and the trafficability of the test case are graded according to a first model, wherein the first model is as follows:
the coverage of the test case is as follows:
wherein: m isoddTotal number of test cases, m, for ODD-based test case sub-libraryfTotal number of test cases, m, for a library of test case sub-libraries based on functional securitysTotal number of test cases, m, for a library of test cases based on expected functional securityeThe total number of test cases of the test case sub-library based on expert experience; n isoddTotal number of real test cases, n, from the ODD-based test case sub-library among the test cases for the actual testfThe total number of real test cases, n, from the function safety-based test case sub-library in the test cases for actual testsThe total number of real test cases, n, from the test case sub-library based on the expected functional safety in the test cases for actual testeThe total number of real test cases derived from the expert experience-based test case sub-library in the test cases for actual testing.
5. The integrated simulation test and evaluation method of the automatic driving system according to claim 4, characterized in that: in the step 5, when the traffic flow library is used for simulation, the state information, the track information and the traffic flow density information of the traffic participants are utilized to carry out simulation evaluation on the traffic conflict of the automatic driving system.
6. The integrated simulation test and evaluation method of the automatic driving system according to claim 5, characterized in that: in the step 5, the traffic conflict is represented by replacing variables of headway, inter-vehicle distance and collision time, when the traffic conflict exists, the occurrence frequency of the traffic conflict is collected to carry out simulation evaluation, the occurrence frequency is the ratio of the number of conflicts to the number of miles driven, the position and the moment when the traffic conflict occurs are extracted, and the time distribution of the traffic conflict and the space distribution of the traffic conflict are formed according to the position and the moment.
7. The integrated simulation test and evaluation method of the automatic driving system according to claim 6, characterized in that: and 6, when the test scene library and the traffic flow library are used for simulation at the same time, carrying out weighted comprehensive evaluation on simulation results under the two conditions to obtain a comprehensive performance safety evaluation value of the safety of the automatic driving automobile.
8. Automatic integrated simulation test of driving system and evaluation system, its characterized in that: the system comprises a simulation interaction module, a modeling module, a database module, a processing module, a scheduling module, a sensor module and a calculation module;
the simulation interaction module is used for acquiring the test requirement and sending the test requirement to the processing module;
the system comprises a modeling module, a traffic flow model and a traffic flow model, wherein the modeling module is used for establishing a vehicle dynamics model of an automatic driving vehicle, the modeling module is used for establishing a scene model, the modeling module is used for establishing a test case, and the modeling module is used for establishing a traffic flow model;
the database module comprises a dynamic model library, a test scene library, a test case library and a traffic flow library which are used for storing the dynamic model library, the test scene library, the test case library and the traffic flow library which are required by the test;
the system comprises a processing module, a database module, a traffic flow model acquisition module, a data processing module and a data processing module, wherein the processing module is used for acquiring a vehicle dynamic model and storing the vehicle dynamic model into a dynamic model library in the database module, the processing module is used for acquiring a scene model and storing the scene model into a test scene library in the database module, the processing module is used for acquiring a test case and storing the test case into a test case library in the database module, and the processing module is used for acquiring a traffic flow model and storing the traffic flow model into a traffic flow library in the database module;
the dispatching module is used for dispatching the dynamic model library, the test scene library, the test case library and the traffic flow library according to the test requirements;
the sensor module is used for detecting a simulation result in a simulation test process;
and the calculation module is used for receiving the simulation result acquired by the processing module from the sensor module, calculating simulation evaluation according to the simulation result and generating an evaluation report.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210100682.8A CN114428998A (en) | 2022-01-27 | 2022-01-27 | Integrated simulation test and evaluation method and system for automatic driving system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210100682.8A CN114428998A (en) | 2022-01-27 | 2022-01-27 | Integrated simulation test and evaluation method and system for automatic driving system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114428998A true CN114428998A (en) | 2022-05-03 |
Family
ID=81313295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210100682.8A Pending CN114428998A (en) | 2022-01-27 | 2022-01-27 | Integrated simulation test and evaluation method and system for automatic driving system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114428998A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115061386A (en) * | 2022-06-27 | 2022-09-16 | 合众新能源汽车有限公司 | Intelligent driving automatic simulation test system and related equipment |
| CN115114786A (en) * | 2022-06-29 | 2022-09-27 | 重庆长安汽车股份有限公司 | Evaluation method, system and storage medium for traffic flow simulation model |
| CN115344503A (en) * | 2022-10-18 | 2022-11-15 | 深圳市城市交通规划设计研究中心股份有限公司 | Traffic flow simulation system and simulation method for automatic driving planning control test |
| CN115576224A (en) * | 2022-11-22 | 2023-01-06 | 中国重汽集团济南动力有限公司 | Simulation test and evaluation method for adaptive cruise control system |
| CN117176611A (en) * | 2023-10-30 | 2023-12-05 | 翼方健数(北京)信息科技有限公司 | Performance test method, system and medium of distributed communication library |
| CN117234192A (en) * | 2023-11-10 | 2023-12-15 | 联友智连科技有限公司 | Intelligent driving domain controller automatic HIL simulation test system and method |
-
2022
- 2022-01-27 CN CN202210100682.8A patent/CN114428998A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115061386A (en) * | 2022-06-27 | 2022-09-16 | 合众新能源汽车有限公司 | Intelligent driving automatic simulation test system and related equipment |
| CN115114786A (en) * | 2022-06-29 | 2022-09-27 | 重庆长安汽车股份有限公司 | Evaluation method, system and storage medium for traffic flow simulation model |
| CN115344503A (en) * | 2022-10-18 | 2022-11-15 | 深圳市城市交通规划设计研究中心股份有限公司 | Traffic flow simulation system and simulation method for automatic driving planning control test |
| CN115576224A (en) * | 2022-11-22 | 2023-01-06 | 中国重汽集团济南动力有限公司 | Simulation test and evaluation method for adaptive cruise control system |
| CN115576224B (en) * | 2022-11-22 | 2023-10-10 | 中国重汽集团济南动力有限公司 | Simulation test and evaluation method for adaptive cruise control system |
| CN117176611A (en) * | 2023-10-30 | 2023-12-05 | 翼方健数(北京)信息科技有限公司 | Performance test method, system and medium of distributed communication library |
| CN117176611B (en) * | 2023-10-30 | 2024-01-30 | 翼方健数(北京)信息科技有限公司 | Performance test method, system and medium of distributed communication library |
| CN117234192A (en) * | 2023-11-10 | 2023-12-15 | 联友智连科技有限公司 | Intelligent driving domain controller automatic HIL simulation test system and method |
| CN117234192B (en) * | 2023-11-10 | 2024-01-30 | 联友智连科技有限公司 | Intelligent driving domain controller automatic HIL simulation test system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114428998A (en) | Integrated simulation test and evaluation method and system for automatic driving system | |
| Amersbach et al. | Defining required and feasible test coverage for scenario-based validation of highly automated vehicles | |
| Tarko | Measuring road safety with surrogate events | |
| Feng et al. | Safety assessment of highly automated driving systems in test tracks: A new framework | |
| Essa et al. | Simulated traffic conflicts: do they accurately represent field-measured conflicts? | |
| GB2544634A (en) | Testbed for lane boundary detection in virtual driving environment | |
| Xu et al. | Estimating project-level vehicle emissions with Vissim and MOVES-matrix | |
| CN114514524A (en) | Multi-agent simulation | |
| Farah | Modeling drivers' passing duration and distance in a virtual environment | |
| CN112819968A (en) | Test method and device for automatic driving vehicle based on mixed reality | |
| CN113918615A (en) | Simulation-based driving experience data mining model construction method and system | |
| Thal et al. | Incorporating safety relevance and realistic parameter combinations in test-case generation for automated driving safety assessment | |
| Kim et al. | Toward prediction of driver awareness of automotive hazards: Driving-video-based simulation approach | |
| Wu et al. | Clustering of several typical behavioral characteristics of commercial vehicle drivers based on GPS data mining: Case study of highways in China | |
| CN114969092A (en) | Driving simulation system based on unreal engine and simulation experiment method | |
| Samaras et al. | Development of a methodology and tool to evaluate the impact of ICT measures on road transport emissions | |
| Lehmann et al. | Use of a criticality metric for assessment of critical traffic situations as part of SePIA | |
| Guo et al. | Generation of a scenario library for testing driver-automation cooperation safety under cut-in working conditions | |
| CN113239609A (en) | Test system and detection method for target identification and monitoring of monocular camera of intelligent vehicle | |
| Lindenmaier et al. | Object-Level Data-Driven Sensor Simulation for Automotive Environment Perception | |
| WO2023215507A1 (en) | System and method for simulating an environment for autonomous vehicle testing | |
| Grigoropoulos et al. | Using Geodata for Simulating Urban Traffic-Current Research in the Field of Traffic Engineering and Control | |
| CN109284889B (en) | Unmanned vehicle reaction agility evaluation method and device | |
| CN116409322A (en) | Virtual track recognition method, device, equipment, storage medium and product | |
| CN110688611A (en) | Method and device for evaluating driving safety of in-service tunnel by environmental factors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |