CN116627793A - Method and device for evaluating authenticity of software in-loop simulation test - Google Patents
Method and device for evaluating authenticity of software in-loop simulation test Download PDFInfo
- Publication number
- CN116627793A CN116627793A CN202310161117.7A CN202310161117A CN116627793A CN 116627793 A CN116627793 A CN 116627793A CN 202310161117 A CN202310161117 A CN 202310161117A CN 116627793 A CN116627793 A CN 116627793A
- Authority
- CN
- China
- Prior art keywords
- data
- test
- index
- simulation test
- evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 393
- 238000004088 simulation Methods 0.000 title claims abstract description 213
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000011156 evaluation Methods 0.000 claims abstract description 142
- 238000004422 calculation algorithm Methods 0.000 claims description 23
- 230000008569 process Effects 0.000 claims description 21
- 230000001133 acceleration Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003068 static effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012417 linear regression Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3664—Environments for testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Educational Administration (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Computer Hardware Design (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to the technical field of vehicle testing, in particular to a method and a device for evaluating the authenticity of software in-loop simulation testing, wherein the method comprises the following steps: acquiring a target test scene of the software in-loop simulation test; in a target test scene, respectively performing software on-loop simulation test and real vehicle test by using a preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculating index errors of each evaluation index according to the simulation test data and the real vehicle test data; and carrying out weighted summation according to the index error of each evaluation index and the corresponding weight coefficient to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is larger than a preset value, the evaluation of the authenticity of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the authenticity of the software in-loop simulation test is judged to be unqualified. Therefore, the problems of poor evaluation effectiveness and accuracy and the like caused by single evaluation dimension of the authenticity of the software in the loop simulation test are solved.
Description
Technical Field
The application relates to the technical field of vehicle testing, in particular to a method and a device for evaluating the authenticity of software in-loop simulation testing.
Background
SIL (software in-the-loop) simulation test is an indispensable technical means for automobile research and development, manufacturing, verification test and other links, and can effectively shorten the technology and product development period and reduce the research and development cost; with the development of the intelligent and networking trend of automobiles, the SIL simulation test technology has a larger playing space, such as simulation test verification of an automatic driving system. SIL simulation test is a key ring for realizing high-order automatic driving floor application, and vehicles with automatic driving function must go through a large number of simulation tests to avoid complex scenes, dangerous scenes and the like which cannot be covered by real vehicle road tests.
As the importance of SIL simulation testing in the field of autopilot testing increases, the assessment of its authenticity is also particularly important. The authenticity of the SIL simulation test directly affects the effectiveness of the SIL simulation test for the autopilot algorithm.
However, the evaluation method of the reality of the automatic driving simulation test in the related technology mainly improves and evaluates the reality of the simulation test in terms of the behavior and actions of traffic participants and the construction of static map scenes, thereby improving the accuracy of the simulation test. From the perspective of the whole test link of SIL simulation test, the existing method has the following limitations: the SIL simulation test has single actual degree evaluation dimension, and the effectiveness and accuracy of the simulation test cannot be directly judged.
Disclosure of Invention
The application provides a method for evaluating the authenticity of software in-loop simulation test, which aims to solve the problems of poor evaluation effectiveness and accuracy and the like caused by single dimension of the authenticity evaluation of the software in-loop simulation test in the related technology.
An embodiment of a first aspect of the present application provides a method for evaluating the authenticity of software in-loop simulation test, including the following steps: acquiring a target test scene of the software in-loop simulation test; under the target test scene, respectively performing software on-loop simulation test and real vehicle test by using a preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculating index errors of each evaluation index according to the simulation test data and the real vehicle test data; and carrying out weighted summation according to the index error of each evaluation index and the corresponding weight coefficient to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is larger than a preset value, the evaluation of the authenticity of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the authenticity of the software in-loop simulation test is judged to be unqualified.
According to the technical means, the real vehicle test and the SIL simulation test are carried out on the acquired test scene, and the corresponding evaluation indexes are extracted, so that the reality of the software in-loop simulation test can be comprehensively evaluated through the evaluation indexes obtained by the real vehicle test and the simulation test, the evaluation dimension is increased, and the reality and the effectiveness of the simulation test are effectively improved.
Optionally, the calculating the index error of each evaluation index according to the simulation test data and the real vehicle test data includes: if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage; if the index error is an error of curve data or continuous data, calculating a correlation coefficient of simulation test data and real vehicle test data, matching the similarity degree grade of the curve data or continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
According to the technical means, the embodiment of the application can calculate the corresponding index error according to different types of index errors, and can improve the effectiveness of simulation test.
Optionally, in the target test scenario, performing software on-loop simulation test and real vehicle test by using a preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes, where the method includes: if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in a test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data; if the evaluation index is a related index of a preset algorithm input dimension, lane line data and target object data are extracted in a test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data; and if the evaluation index is a related index of a preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of the one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
According to the technical means, the embodiment of the application evaluates the fidelity through three dimensions of scene fidelity, algorithm input and vehicle performance, and improves the effectiveness and reliability of simulation test.
Optionally, in the target test scenario, before the loop simulation test and the real vehicle test, respectively, the software performing the loop simulation test and the real vehicle test by using a preset test strategy further includes: and respectively reproducing the target test scene in a real environment and a simulation environment to respectively perform software on-loop simulation test and real vehicle test.
According to the technical means, the embodiment of the application needs to perform environment reproduction before the software is subjected to the ring simulation test and the real vehicle test, so that the reliability of the simulation test is ensured, and the reality is increased.
An embodiment of a second aspect of the present application provides a device for evaluating the authenticity of software in-loop simulation test, including: the acquisition module is used for acquiring a target test scene of the software in-loop simulation test; the calculation module is used for respectively carrying out software in-loop simulation test and real vehicle test by utilizing a preset test strategy under the target test scene to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculating index errors of each evaluation index according to the simulation test data and the real vehicle test data; and the judging module is used for carrying out weighted summation according to the index error of each evaluation index and the corresponding weight coefficient to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is larger than a preset value, the evaluation of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the software in-loop simulation test is judged to be unqualified.
Optionally, the computing module is further configured to: if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage; if the index error is an error of curve data or continuous data, calculating a correlation coefficient of simulation test data and real vehicle test data, matching the similarity degree grade of the curve data or continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
Optionally, the computing module is further configured to: if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in a test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data; if the evaluation index is a related index of a preset algorithm input dimension, lane line data and target object data are extracted in a test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data; and if the evaluation index is a related index of a preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of the one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
Optionally, the method further comprises: and the reproduction module is used for respectively reproducing the target test scene in a real environment and a simulation environment before the software in-loop simulation test and the real vehicle test by utilizing a preset test strategy under the target test scene so as to respectively perform the software in-loop simulation test and the real vehicle test.
An embodiment of a third aspect of the present application provides an electronic device, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to realize the method for evaluating the authenticity of the software in-loop simulation test.
An embodiment of a fourth aspect of the present application provides a computer-readable storage medium having stored thereon a computer program for execution by a processor for implementing the method for evaluating the authenticity of a software in-loop simulation test as described in the above embodiment.
Therefore, the application has at least the following beneficial effects:
(1) According to the embodiment of the application, the real vehicle test and the SIL simulation test are carried out on the acquired test scene, and the corresponding evaluation indexes are extracted, so that the actual degree of the software in-loop simulation test can be comprehensively evaluated through the evaluation indexes obtained by the real vehicle test and the simulation test, the evaluation dimension is increased, and the actual degree and the effectiveness of the simulation test are effectively improved.
(2) The embodiment of the application can calculate the corresponding index error according to the types of different index errors, and can improve the effectiveness of the simulation test.
(3) According to the embodiment of the application, the fidelity is evaluated through three dimensions of scene fidelity, algorithm input and vehicle performance, so that the effectiveness and reliability of simulation test are improved.
(4) According to the embodiment of the application, the environment reproduction is needed before the loop simulation test and the real vehicle test of the software, so that the reliability of the simulation test is ensured, and the reality is increased.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flowchart of a method for evaluating the authenticity of a software in-loop simulation test according to an embodiment of the present application;
fig. 2 is a schematic diagram of a front car cut-in scene provided according to an embodiment of the present application;
FIG. 3 is a flowchart of a scene fidelity assessment method according to an embodiment of the present application;
FIG. 4 is a flow chart of an algorithm input consistency assessment method provided according to an embodiment of the present application;
FIG. 5 is a flowchart of a vehicle performance consistency assessment method provided in accordance with an embodiment of the present application;
FIG. 6 is a block diagram of a SIL simulation test fidelity assessment system provided in accordance with an embodiment of the present application;
FIG. 7 is a flowchart of a method for evaluating the authenticity of a software in-loop simulation test according to an embodiment of the present application;
FIG. 8 is a block diagram illustrating a device for evaluating the authenticity of a software in-loop simulation test according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The method for evaluating the authenticity of the software in-loop simulation test according to the embodiment of the application is described below with reference to the accompanying drawings. Aiming at the problems that the simulation test authenticity assessment dimension mentioned in the background art is single, the validity and the accuracy of the simulation test cannot be directly judged, and a method for systematically assessing the authenticity is lacked, the application provides a method for assessing the authenticity of software in a loop simulation test, wherein in the method, a real vehicle test and an SIL simulation test are respectively carried out in an acquired test scene, and the authenticity is assessed in three dimensions of scene fidelity, algorithm input and vehicle representation. Therefore, the problems of poor evaluation effectiveness and accuracy and the like caused by single evaluation dimension of the authenticity of the software in the loop simulation test in the related technology are solved.
Specifically, fig. 1 is a schematic flow chart of a method for evaluating the authenticity of a software in-loop simulation test according to an embodiment of the present application.
As shown in fig. 1, the method for evaluating the authenticity of the software in the loop simulation test comprises the following steps:
in step S101, a target test scenario of the software in-loop simulation test is acquired.
The target test scene may be a typical test scene acquired through natural driving data.
It should be noted that, the typical test scenario is that a typical scenario including, but not limited to, NGSIM data set or long-term security road test data set is obtained by performing scene screening, such as cut-in of a front car, crossing of a pedestrian, cut-out of a front car, automatic following of a car, etc., and static road network data and dynamic traffic participant data of the typical test scenario are extracted and recorded.
For example, taking a previous vehicle cut scene as an example, static road network data (including road length, number of lanes, lane width, lane line type) and dynamic traffic participant data (from initial position, initial speed, previous vehicle initial position, initial speed, cut-in action duration, vehicle speed after cut-in) are extracted, and a scene diagram is shown in fig. 2.
In step S102, in the target test scenario, the software in-loop simulation test and the real vehicle test are performed respectively by using a preset test strategy, so as to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and an index error of each evaluation index is calculated according to the simulation test data and the real vehicle test data.
The preset test strategy refers to a method for performing software on-loop simulation test and real vehicle test to acquire required data.
The evaluation index can comprise relevant objective indexes of three dimensions of scene fidelity, algorithm input and vehicle performance.
In the embodiment of the application, under the target test scene, before the loop simulation test and the real vehicle test, the software is respectively carried out by utilizing the preset test strategy, and the method further comprises the following steps: and respectively reproducing the target test scene in the real environment and the simulation environment to respectively perform the software on-loop simulation test and the real vehicle test.
It can be understood that the test scene is reproduced and tested in the real environment and the simulation environment, and relevant objective index data of three dimensions of scene fidelity, algorithm input and vehicle expression are extracted, the test scene is built in the real environment and the simulation environment (including but not limited to VTD simulation software) based on static map data and dynamic traffic participant data of the scene, and the test is respectively carried out.
In the embodiment of the application, in a target test scene, a software in-loop simulation test and a real vehicle test are respectively carried out by utilizing a preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes, wherein the method comprises the following steps: if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in the test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data; if the evaluation index is a correlation index of a preset algorithm input dimension, lane line data and target object data are extracted in the test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data; and if the evaluation index is a related index of the preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
It can be understood that when the software is subjected to the ring simulation test and the real vehicle test by utilizing the preset management strategy, different simulation test data and real vehicle test data are extracted according to different evaluation indexes. When the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object, such as speed, distance, action time, track curve and the like; when the evaluation index is a related index of a preset algorithm input dimension, extracting lane line data (lane line equation coefficient) and target object data (the transverse and longitudinal speeds of a target object and the transverse and longitudinal relative distance between the target object and a vehicle); and when the evaluation index is a related index of the preset vehicle performance dimension, extracting data such as transverse and longitudinal speed, transverse and longitudinal acceleration, vehicle following or stopping distance and the like.
For example, taking a previous car cut scene as an example, the running data of the target is extracted in the Jing Baozhen degree dimension, as shown in table 1, where table 1 is a scene fidelity dimension objective index table:
TABLE 1
Data such as a lane, a relative position relation between a vehicle and a front vehicle and the like are extracted in an algorithm input dimension, as shown in table 2, wherein table 2 is an algorithm input dimension objective index:
TABLE 2
| Parameter name | (symbol) |
| Left lane line equation | f_l(x,y) |
| Right lane line equation | f_r(x,y) |
| The center line pressing time of the rear axle of the front vehicle is the longitudinal distance from the vehicle to the front vehicle | Dis |
Data such as the transverse and longitudinal speed, the transverse and longitudinal acceleration, the following distance or the stopping distance of the vehicle are extracted in the vehicle performance dimension, as shown in table 3, wherein table 3 is an objective index table of the vehicle performance dimension:
TABLE 3 Table 3
| Parameter name | (symbol) |
| Longitudinal acceleration of the bicycle at the central line pressing moment of the rear axle of the front bicycle | a_x |
| Front car cut-in is stable, and then the speed of the self-car is fast | V_x |
| Distance between the front vehicle and the following vehicle after the front vehicle is cut into stably | D_x |
In the embodiment of the application, calculating the index error of each evaluation index according to the simulation test data and the real vehicle test data comprises the following steps: if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage; if the index error is the error of the curve data or the continuous data, calculating the correlation coefficient of the simulation test data and the real vehicle test data, matching the similarity degree grade of the curve data or the continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
It will be appreciated that the index error for each evaluation index is determined based on the different types of index errors. Aiming at the error of the objective variable index, calculating the error percentage of the index variable value of the simulation test and the index variable of the real vehicle test; for curve data or continuous data, a correlation coefficient of simulation test data and real vehicle test data is calculated by using a linear regression method, so that the similarity degree of two sections of curve data is judged.
The error percentage of the index variable value of the simulation test and the index variable of the real vehicle test is calculated aiming at the error of the objective variable index, and the formula is thatρ is the consistency percentage of the simulation test and the real vehicle test in terms of objective indexes; xi is the value of the objective index in the simulation test; x0 is the value of the objective index in the real vehicle test.
And calculating the correlation coefficient of the simulation test data and the real vehicle test data by using a linear regression method aiming at the curve data or the continuous data, so as to judge the similarity degree of the two sections of curve data. The method comprises the following steps:
A. taking the x data group with the same interval and the value step length, and calculating the y data group under the curve equation in the simulation test and the real vehicle test;
B. based on two groups y And (3) carrying out linear regression fitting on the data group, and calculating a correlation coefficient R, wherein R can be used as the similarity degree of two curve data.
Based on the above method, the consistency results of the three objective indexes with different dimensions can be calculated respectively as shown in the following table 4, wherein table 4 is an objective index consistency result table:
TABLE 4 Table 4
In step S103, weighted summation is performed according to the index error of each evaluation index and the corresponding weight coefficient, so as to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is greater than a preset value, the evaluation of the authenticity of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the authenticity of the software in-loop simulation test is judged to be unqualified.
Wherein the weight ratio of each evaluation index is calculated by a weight coefficient of three evaluation dimensions including but not limited to analytic hierarchy process. The weight coefficients of the evaluation dimensions are denoted as A1, A2, A3 and 9 indexes, and the weight coefficients of the evaluation dimensions are denoted as A1, A2, A3, a4, a5, a6, a7, a8 and a9.
It can be understood that, based on the error percentage of the objective index of each evaluation dimension and the weight coefficient of each evaluation dimension, the fidelity result of the SIL simulation test is calculated according to the following calculation formula:
ρ＝[A1·(a1·ρ_1+a1·ρ_2+a1·ρ_3)+A2·(a4·ρ_4+a5·ρ_5+a6·ρ_6)+A3·(a7·ρ_7+a8·ρ_8+a9·ρ_9)]
specifically, the scene fidelity evaluation flow includes: extracting data required by scene fidelity according to the real vehicle test data and the simulation test data, performing weight calculation, and evaluating the scene fidelity, as shown in fig. 3; the algorithm input evaluation flow comprises the following steps: according to the real vehicle test data and the simulation test data, the required data is input by an extraction algorithm, weight calculation is performed, and the consistency of the input algorithm is evaluated, as shown in fig. 4; the vehicle performance evaluation flow includes: and extracting data required by vehicle performance according to the real vehicle test data and the simulation test data, performing weight calculation, and evaluating vehicle performance consistency, as shown in fig. 5.
In the embodiment of the application, a set of simulation test authenticity judging system is also provided, which is used for judging whether the authenticity evaluation of the software in the loop simulation test is qualified or not, and comprises the following steps: the system comprises a data input interface, a scene fidelity computing module, a simulation algorithm input consistency computing module, a vehicle performance consistency computing module, a SIL simulation test fidelity result computing module and a radar chart display interface of fidelity results of different dimensions. The evaluation system is a system for calculating the fidelity result of SIL simulation test based on the acquired real vehicle test data and simulation test data and based on a set of automatic loading test data including, but not limited to, matlab software developed in a PC (Personal Computer ) computer, and the framework of the evaluation system is shown in FIG. 6.
The following describes a method for evaluating the authenticity of the software in-loop simulation test according to a specific embodiment, as shown in fig. 7, and the steps are as follows:
step 1, acquiring a typical test scene based on natural driving data;
step 2, reproducing the test scene in a closed road and a simulation environment, testing, and extracting relevant objective index data of three dimensions of scene fidelity, algorithm input and vehicle expression;
step 3, calculating objective index errors of the real vehicle test and the simulation test;
step 4, determining weights of three evaluation dimensions;
and 5, calculating and outputting a SIL simulation test fidelity result.
According to the method for evaluating the authenticity of the software in-loop simulation test, which is provided by the embodiment of the application, the real vehicle test and the SIL simulation test are carried out on the acquired test scene, and the corresponding evaluation index is extracted, so that the authenticity of the software in-loop simulation test can be comprehensively evaluated through the evaluation indexes obtained by the real vehicle test and the simulation test, the evaluation dimension is increased, and the authenticity and the effectiveness of the simulation test are effectively improved; corresponding index errors can be calculated according to different types of index errors, and effectiveness of simulation test can be improved; the reality is evaluated through three dimensions of scene fidelity, algorithm input and vehicle expression, so that the effectiveness and reliability of simulation test are improved; the environment reproduction is needed before the software is subjected to the loop simulation test and the real vehicle test, so that the reliability of the simulation test is ensured, and the reality is increased.
The embodiment of the application provides a device for evaluating the authenticity of the software in-loop simulation test.
FIG. 8 is a block diagram of a software in-loop simulation test authenticity assessment device according to an embodiment of the present application.
As shown in fig. 8, the software in-loop simulation test authenticity assessment device 10 includes: the device comprises an acquisition module 100, a calculation module 200 and a judgment module 300.
The acquiring module 100 is configured to acquire a target test scenario of an in-loop simulation test of software; the calculation module 200 is configured to perform a software-in-loop simulation test and a real vehicle test respectively by using a preset test strategy under a target test scenario, obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculate an index error of each evaluation index according to the simulation test data and the real vehicle test data; the judging module 300 is configured to perform weighted summation according to the index error of each evaluation index and the corresponding weight coefficient, so as to obtain an evaluation value of the software in the loop simulation test, where if the evaluation value is greater than a preset value, the evaluation of the authenticity of the software in the loop simulation test is qualified, and if not, the evaluation of the authenticity of the software in the loop simulation test is unqualified.
In an embodiment of the present application, the computing module 200 is further configured to: if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage; if the index error is the error of the curve data or the continuous data, calculating the correlation coefficient of the simulation test data and the real vehicle test data, matching the similarity degree grade of the curve data or the continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
In an embodiment of the present application, the computing module 200 is further configured to: if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in the test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data; if the evaluation index is a correlation index of a preset algorithm input dimension, lane line data and target object data are extracted in the test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data; and if the evaluation index is a related index of the preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
In the embodiment of the present application, the apparatus 10 of the embodiment of the present application further includes: and a reproduction module. The reproduction module is used for respectively reproducing the target test scene in the real environment and the simulation environment before the software in-loop simulation test and the real vehicle test by utilizing the preset test strategy under the target test scene so as to respectively perform the software in-loop simulation test and the real vehicle test.
It should be noted that the explanation of the embodiment of the method for evaluating the authenticity of the software in-loop simulation test is also applicable to the apparatus for evaluating the authenticity of the software in-loop simulation test in this embodiment, and will not be repeated here.
According to the device for evaluating the authenticity of the software in-loop simulation test, which is provided by the embodiment of the application, the real vehicle test and the SIL simulation test are carried out on the acquired test scene, and the corresponding evaluation index is extracted, so that the authenticity of the software in-loop simulation test can be comprehensively evaluated through the evaluation indexes obtained by the real vehicle test and the simulation test, the evaluation dimension is increased, and the authenticity and the effectiveness of the simulation test are effectively improved; corresponding index errors can be calculated according to different types of index errors, and effectiveness of simulation test can be improved; the reality is evaluated through three dimensions of scene fidelity, algorithm input and vehicle expression, so that the effectiveness and reliability of simulation test are improved; the environment reproduction is needed before the software is subjected to the loop simulation test and the real vehicle test, so that the reliability of the simulation test is ensured, and the reality is increased.
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
memory 901, processor 902, and a computer program stored on memory 901 and executable on processor 902.
The processor 902 implements the method for evaluating the authenticity of the software provided in the above embodiment in the loop simulation test when executing the program.
Further, the electronic device further includes:
a communication interface 903 for communication between the memory 901 and the processor 902.
Memory 901 for storing a computer program executable on processor 902.
The memory 901 may include a high-speed RAM (Random Access Memory ) memory, and may also include a nonvolatile memory, such as at least one magnetic disk memory.
If the memory 901, the processor 902, and the communication interface 903 are implemented independently, the communication interface 903, the memory 901, and the processor 902 may be connected to each other through a bus and perform communication with each other. The bus may be an ISA (Industry Standard Architecture ) bus, a PCI (Peripheral Component, external device interconnect) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 9, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 901, the processor 902, and the communication interface 903 are integrated on a chip, the memory 901, the processor 902, and the communication interface 903 may communicate with each other through internal interfaces.
The processor 902 may be a CPU (Central Processing Unit ) or ASIC (Application Specific Integrated Circuit, application specific integrated circuit) or one or more integrated circuits configured to implement embodiments of the present application.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the method for evaluating the authenticity of the software in-loop simulation test as above.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable gate arrays, field programmable gate arrays, and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (10)
1. The method for evaluating the authenticity of the software in-loop simulation test is characterized by comprising the following steps of:
acquiring a target test scene of the software in-loop simulation test;
under the target test scene, respectively performing software on-loop simulation test and real vehicle test by using a preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculating index errors of each evaluation index according to the simulation test data and the real vehicle test data;
and carrying out weighted summation according to the index error of each evaluation index and the corresponding weight coefficient to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is larger than a preset value, the evaluation of the authenticity of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the authenticity of the software in-loop simulation test is judged to be unqualified.
2. The method of claim 1, wherein calculating an index error for each evaluation index from the simulated test data and the real vehicle test data comprises:
if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage;
if the index error is an error of curve data or continuous data, calculating a correlation coefficient of simulation test data and real vehicle test data, matching the similarity degree grade of the curve data or continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
3. The method of claim 1, wherein the performing, in the target test scenario, the software-in-loop simulation test and the real vehicle test with the preset test strategy to obtain simulation test data and real vehicle test data of one or more evaluation indexes includes:
if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in a test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data;
if the evaluation index is a related index of a preset algorithm input dimension, lane line data and target object data are extracted in a test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data;
and if the evaluation index is a related index of a preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of the one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
4. A method according to any one of claims 1-3, wherein in the target test scenario, software is performed before the loop simulation test and the real vehicle test, respectively, using a preset test strategy, further comprising:
and respectively reproducing the target test scene in a real environment and a simulation environment to respectively perform software on-loop simulation test and real vehicle test.
5. A device for evaluating the authenticity of a software in-loop simulation test, comprising:
the acquisition module is used for acquiring a target test scene of the software in-loop simulation test;
the calculation module is used for respectively carrying out software in-loop simulation test and real vehicle test by utilizing a preset test strategy under the target test scene to obtain simulation test data and real vehicle test data of one or more evaluation indexes, and calculating index errors of each evaluation index according to the simulation test data and the real vehicle test data;
and the judging module is used for carrying out weighted summation according to the index error of each evaluation index and the corresponding weight coefficient to obtain an evaluation value of the software in-loop simulation test, wherein if the evaluation value is larger than a preset value, the evaluation of the software in-loop simulation test is judged to be qualified, otherwise, the evaluation of the software in-loop simulation test is judged to be unqualified.
6. The apparatus of claim 5, wherein the computing module is further to:
if the index error is the error of the objective variable index, calculating the error percentage of the simulation test data and the real vehicle test data, and determining the index error of each evaluation index based on the error percentage;
if the index error is an error of curve data or continuous data, calculating a correlation coefficient of simulation test data and real vehicle test data, matching the similarity degree grade of the curve data or continuous data by using the correlation coefficient, and determining the index error of each evaluation index based on the similarity degree grade.
7. The apparatus of claim 5, wherein the computing module is further to:
if the evaluation index is a related index of a preset scene fidelity dimension, extracting operation data of a target object in a test process, and generating simulation test data and real vehicle test data of one or more evaluation indexes based on the operation data;
if the evaluation index is a related index of a preset algorithm input dimension, lane line data and target object data are extracted in a test process, and simulation test data and real vehicle test data of one or more evaluation indexes are generated based on the lane line data and the target object data;
and if the evaluation index is a related index of a preset vehicle performance dimension, extracting transverse and longitudinal speed data, transverse and longitudinal acceleration data and vehicle following or stopping distance data in the test process, and generating simulation test data and real vehicle test data of the one or more evaluation indexes based on the transverse and longitudinal speed data, the transverse and longitudinal acceleration data and the vehicle following or stopping distance data.
8. The apparatus according to any one of claims 5-7, further comprising:
and the reproduction module is used for respectively reproducing the target test scene in a real environment and a simulation environment before the software in-loop simulation test and the real vehicle test by utilizing a preset test strategy under the target test scene so as to respectively perform the software in-loop simulation test and the real vehicle test.
9. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method of evaluating the realism of a software in-loop simulation test as claimed in any one of claims 1 to 4.
10. A computer readable storage medium having stored thereon a computer program, characterized in that the program is executed by a processor for implementing a method of evaluating the authenticity of a software in-loop simulation test according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310161117.7A CN116627793A (en) | 2023-02-23 | 2023-02-23 | Method and device for evaluating authenticity of software in-loop simulation test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310161117.7A CN116627793A (en) | 2023-02-23 | 2023-02-23 | Method and device for evaluating authenticity of software in-loop simulation test |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116627793A true CN116627793A (en) | 2023-08-22 |
Family
ID=87640542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310161117.7A Pending CN116627793A (en) | 2023-02-23 | 2023-02-23 | Method and device for evaluating authenticity of software in-loop simulation test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116627793A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117095239A (en) * | 2023-10-16 | 2023-11-21 | 光轮智能(北京)科技有限公司 | Simulation asset authenticity evaluation method, control device and storage medium |
| CN117217422A (en) * | 2023-11-07 | 2023-12-12 | 国汽(北京)智能网联汽车研究院有限公司 | Vehicle motion control capability assessment method, system, device and medium thereof |
-
2023
- 2023-02-23 CN CN202310161117.7A patent/CN116627793A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117095239A (en) * | 2023-10-16 | 2023-11-21 | 光轮智能(北京)科技有限公司 | Simulation asset authenticity evaluation method, control device and storage medium |
| CN117095239B (en) * | 2023-10-16 | 2023-12-15 | 光轮智能(北京)科技有限公司 | Simulation asset authenticity evaluation method, control device and storage medium |
| CN117217422A (en) * | 2023-11-07 | 2023-12-12 | 国汽(北京)智能网联汽车研究院有限公司 | Vehicle motion control capability assessment method, system, device and medium thereof |
| CN117217422B (en) * | 2023-11-07 | 2024-03-22 | 国汽(北京)智能网联汽车研究院有限公司 | Vehicle motion control capability assessment method, system, device and medium thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116627793A (en) | Method and device for evaluating authenticity of software in-loop simulation test | |
| Xia et al. | Automatic generation method of test scenario for ADAS based on complexity | |
| CN115099051A (en) | Automatic driving simulation test scene generation method and device, vehicle and storage medium | |
| CN115544637A (en) | Vehicle-road cooperative multi-dimensional performance evaluation test method and device | |
| US20180017402A1 (en) | Method and system for vehicle speed profile generation | |
| CN116562175B (en) | Method for evaluating influence of vehicle configuration on safety of automatic driving system | |
| CN115469635A (en) | Verification method, device and equipment for safety of automatic driving simulation test function | |
| CN109711043A (en) | A kind of deformable mobile obstacle collision simulation method and computer storage medium based on CAE | |
| CN118037117A (en) | Commercial vehicle performance evaluation method, system and equipment | |
| CN111767630A (en) | Virtual simulation test method and device applied to intelligent driving | |
| CN115100757B (en) | Method and device for storing automobile data, vehicle and storage medium | |
| CN116244974A (en) | Method and device for evaluating authenticity of simulated traffic flow data | |
| CN115146491B (en) | Test method and device for automatic driving system, storage medium and electronic device | |
| CN115964800A (en) | Method and device for vehicle AEB calibration, electronic equipment and storage medium | |
| Gavrilov et al. | Method of mathematical description for digital system blocks logical models | |
| CN114035545A (en) | Automatic driving algorithm simulation test method and device and electronic equipment | |
| WO2023097519A1 (en) | Scenario acquisition method and apparatus | |
| CN114544201B (en) | Method and device for evaluating vehicle braking performance, vehicle and storage medium | |
| CN112328477B (en) | Method and device for generating test case of automatic driving algorithm and electronic equipment | |
| CN114580213B (en) | Multi-stage road side simulation method, device, equipment and storage medium | |
| CN115447616B (en) | Method and device for generating objective index of vehicle driving | |
| CN113109055B (en) | Test scene grade calculation method and device and computer storage medium | |
| CN117313426A (en) | Method, device, equipment and storage medium for generating vehicle data | |
| CN115221679A (en) | HARA quantitative analysis method and system for automatic driving system | |
| CN115470616A (en) | Vehicle simulation test method, apparatus, computer device, medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |