CN118032374A - Test evaluation method, computer storage medium and terminal - Google Patents
Test evaluation method, computer storage medium and terminal Download PDFInfo
- Publication number
- CN118032374A CN118032374A CN202410336952.4A CN202410336952A CN118032374A CN 118032374 A CN118032374 A CN 118032374A CN 202410336952 A CN202410336952 A CN 202410336952A CN 118032374 A CN118032374 A CN 118032374A
- Authority
- CN
- China
- Prior art keywords
- sampling
- function
- scene
- test
- equivalence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 150
- 238000011156 evaluation Methods 0.000 title claims abstract description 34
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000005070 sampling Methods 0.000 claims abstract description 103
- 238000012545 processing Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 79
- 238000000034 method Methods 0.000 claims description 42
- 238000004590 computer program Methods 0.000 claims description 12
- 238000009826 distribution Methods 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 2
- 238000011076 safety test Methods 0.000 abstract description 5
- 238000012614 Monte-Carlo sampling Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000003247 decreasing effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000000342 Monte Carlo simulation Methods 0.000 description 3
- 238000010998 test method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000009781 safety test method Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007430 reference method Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a test evaluation method, a computer storage medium and a terminal, comprising the following steps: determining a coverage area of each sampling scene for a group of sampling scenes; determining a weight function of the group of sampling scenes according to the coverage range of each sampling scene; determining an estimated value of the test index according to the determined weight function of the set of sampling scenes; obtaining a macroscopic equivalent function according to the theoretical value and the estimated value of the test index; processing errors of the automatic driving agent model and the automatic driving real model to obtain a microcosmic equivalence function; determining a comprehensive equivalence function of the equivalence test evaluation according to the macroscopic equivalence function and the microscopic equivalence function; and carrying out iterative operation on the comprehensive equivalence function to obtain an optimal sampling scene. According to the embodiment of the disclosure, the sampling scene of a small amount of representative test is generated by measuring the scene coverage through the comprehensive equivalence function, so that the reliability of the automatic driving safety test under a small amount of test cost is ensured.
Description
Technical Field
The present application relates to, but is not limited to, autopilot technology, and relates to a method, computer storage medium and terminal for test evaluation.
Background
The current trial-and-error operation of the automatic driving automobile at home and abroad faces remarkable safety problems, and the large-scale landing of the automatic driving technology is hindered. Some foreign companies automatically drive open roads to run and then accident occurs, so that the operation license is destroyed, and the companies face difficulties; thus, reliable safety testing is an essential element before autopilot is put into use. The open road test is a common safety test method, and because a large number of low-risk scenes exist in the real world, the open road test is difficult to effectively measure the performance of automatic driving in the dangerous scenes, and the test efficiency is low; furthermore, performing tests on real roads may present a safety risk to society.
The iteration period of automatic driving research and development is short, the algorithm is updated rapidly, and large-scale test is difficult to carry out; government and third party testing agencies are also difficult to conduct large scale tests on a variety of autopilot algorithms. Using the method in the related art, the result of performing the test under the limitation of a small number of test times is unreliable; the scene library containing the key scenes is generated in advance based on expert experience, a small number of scenes are selected for testing, and the performance of automatic driving under dangerous conditions can be estimated, but the method is difficult to quantify the test result due to the fact that experience is needed, or the test result does not have theoretical reliability guarantee.
In conclusion, how to theoretically ensure the reliability of the automatic driving safety test under a small amount of test cost becomes a problem to be solved.
Disclosure of Invention
The following is a summary of the subject matter of the detailed description of the application. This summary is not intended to limit the scope of the claims.
The embodiment of the disclosure provides a test evaluation method, a computer storage medium and a terminal, which can ensure the reliability of automatic driving safety test under a small test cost.
The embodiment of the disclosure provides a test evaluation method, which comprises the following steps:
Determining a coverage area of each sampling scene for a group of sampling scenes;
Determining a weight function of the group of sampling scenes according to the coverage range of each sampling scene;
determining an estimated value of the test index according to the determined weight function of the set of sampling scenes;
obtaining a macroscopic equivalent function according to the theoretical value of the test index and the determined estimated value;
processing errors of the automatic driving agent model and the automatic driving real model to obtain a microcosmic equivalence function;
determining a comprehensive equivalence function of the equivalence test evaluation according to the obtained macroscopic equivalence function and microscopic equivalence function;
Performing iterative operation on the determined comprehensive equivalence function to obtain an optimal sampling scene;
Wherein the set of sampling scenes comprises n sampling scenes, and n is smaller than or equal to a preset numerical value; the microcosmic equivalence function represents the difference of test indexes of the sampling scene x i and other sampling scenes in the coverage range of the sampling scene x i, and x i represents the ith sampling scene in n sampling scenes; the macroscopic equivalence function represents the difference between the estimated value and the theoretical value of the automatic driving integral test index obtained through all sampling scenes.
In another aspect, embodiments of the present disclosure also provide a computer storage medium having a computer program stored therein, which when executed by a processor, implements the method of test evaluation described above.
In still another aspect, an embodiment of the present disclosure further provides a terminal, including: a memory and a processor, the memory storing a computer program; wherein,
The processor is configured to execute the computer program in the memory;
the computer program, when executed by the processor, implements a method of test evaluation as described above.
Compared with the related art, the method and the device for testing the automatic driving safety have the advantages that the comprehensive equivalence function of the equivalent test evaluation is determined through the macroscopic equivalence function and the microscopic equivalence function, the sampling scene of a small amount of representative tests is generated through measuring the scene coverage, and the reliability of the automatic driving safety test under a small amount of test cost is guaranteed.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. Other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The accompanying drawings are included to provide an understanding of the principles of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain, without limitation, the principles of the application.
FIG. 1 is a flow chart of a method of test evaluation according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a cut scene of an autonomous car according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of natural probability distribution of a cut scene according to an embodiment of the disclosure;
FIG. 4 is a diagram illustrating the effect of sampling scene selection and coverage estimation according to an embodiment of the disclosure;
fig. 5 is an effect diagram of another sampling scenario selection and coverage estimation according to an embodiment of the present disclosure.
Detailed Description
The present application has been described in terms of several embodiments, but the description is illustrative and not restrictive, and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the described embodiments. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or in place of any other feature or element of any other embodiment unless specifically limited.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features and elements of the present application may also be combined with any conventional features or elements to form a unique inventive arrangement as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive arrangements to form another unique inventive arrangement as defined in the claims. It is therefore to be understood that any of the features shown and/or discussed in the present application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not to be restricted except in light of the attached claims and their equivalents. Further, various modifications and changes may be made within the scope of the appended claims.
Furthermore, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other sequences of steps are possible as will be appreciated by those of ordinary skill in the art. Accordingly, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. Furthermore, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
FIG. 1 is a flow chart of a method of test evaluation according to an embodiment of the present disclosure, as shown in FIG. 1, comprising:
Step 101, determining coverage range of each sampling scene for a group of sampling scenes; when the embodiment of the disclosure is initialized, a set of initial sampling scenes can be generated by a method of random sampling, quasi-Monte Carlo sampling (RQMC) and the like.
In an illustrative example, prior to step 101, the method of the embodiments of the present disclosure further includes: modeling scene space, determining sampling scene space and probability of sampling scene in natural distribution, and testing index theoretical values; wherein determining the scene space basis modeling comprises: the scene in the sampling scene space is X epsilon X, the occurrence probability of the scene in natural distribution is p (X), and the theoretical value of the test index isWherein A represents an event of interest,/>A test index value representing any scene x within the scene space (e.g., the probability of occurrence of an event of interest a in scene x).
102, Determining a weight function of the group of sampling scenes according to the coverage range of each determined sampling scene;
step 103, determining an estimated value of the test index according to the determined weight function of the set of sampling scenes;
104, obtaining a macroscopic equivalent function according to the theoretical value of the test index and the determined estimated value;
Step 105, processing errors of the automatic driving agent model and the automatic driving real model to obtain a microcosmic equivalence function; the real model in the embodiment of the disclosure is an actual automatic driving model to be tested, and each scene x has a test index The agent model of autopilot is an estimate of the real autopilot model, with test index/>, at each scene xThe following theoretical derivation and analysis is due to the actual autopilot test index/>Unknown before testing, all use/>As an estimate thereof.
Step 106, determining a comprehensive equivalence function of the equivalence test evaluation according to the obtained macroscopic equivalence function and microscopic equivalence function;
step 107, performing iterative operation on the determined objective function to obtain an optimal sampling scene;
Wherein, a group of sampling scenes comprises n sampling scenes, and n is smaller than or equal to a preset numerical value; the microscopic equivalence function represents the difference of the test indexes of the sampling scene x i and other sampling scenes in the coverage range of the sampling scene x i, and x i represents the ith of n sampling scenes; the macroscopic equivalence function represents the difference between the estimated value and the theoretical value of the automatic driving overall test index obtained through all sampling scenes.
According to the embodiment of the disclosure, the comprehensive equivalence function of the equivalent test evaluation is determined through the macroscopic equivalence function and the microscopic equivalence function, a sampling scene of a small amount of representative tests is generated through measuring the scene coverage, the evaluation result equivalent to the full test is obtained through the tests on the small amount of representative scenes, and the reliability of the automatic driving safety test under the small amount of test cost is ensured.
In one illustrative example, an embodiment of the present disclosure determines an estimated value of a test metric from a determined weight function of the set of sampling scenarios, comprising:
and calculating the estimated value of the test index in a weighted sum mode according to the weight function.
In one illustrative example, the weighting function of the disclosed embodiments may also be determined in other ways, such as: a constant weight function, a data-driven learnable weight function according to the importance weight function of priori knowledge;
In an illustrative example, the disclosed embodiment estimates may also be determined in other ways in the related art, such as: general form estimation function The embodiments of the present disclosure are not limited in this regard.
In one illustrative example, the test index of the disclosed embodiments has an estimated value ofThe expression of the estimated value may be:
Wherein A represents an event of interest, Test index value representing sampling scene x i, w (x i) represents weight function satisfaction/>C (x i) represents the coverage of the sampling scene x i.
In one illustrative example, the macroscopic equivalent function of the disclosed embodiments is E 1, the expression of the macroscopic equivalent function is:
wherein mu is the theoretical value of the test index, Is an estimate of the test index.
According to the embodiment E 1 of the disclosure, macroscopic equivalence of a test scene is reflected according to the estimated value and the theoretical value of the test index, namely whether the true value of the tested index is accurately equivalent through a group of small-scale tests or not.
In one illustrative example, embodiments of the present disclosure record a theoretical value of μ for a test indicator (e.g., accident rate) for an autonomous car,Wherein A represents an interest event (such as a collision accident), P (x) represents a natural probability distribution of a test scene x, and P (A|x) is a test index of the interest event A on a sampling scene x (such as the occurrence probability of the event A in the sampling scene x);
The test method in the related art realizes test evaluation based on the Monte Carlo sampling principle, and the test efficiency is low under the condition of extremely low occurrence probability of the event, so that effective test results are difficult to obtain in practical application.
In one illustrative example, the microscopic equivalent function of the disclosed embodiments is E 2, the expression of which is:
Where C (x i) represents the coverage of the sampling scene x i, An estimate representing the error between the proxy model of autopilot and the real model of autopilot, i.e. the test index value/>, using the sampling scenario x i Test index value/>, with other scenes x within its coverage areaAnd p (x) represents the probability of scene x occurring in the natural distribution.
In one illustrative example, the expression of coverage C (x i) of sampling scenario x i of the disclosed embodiment is:
Where d (x, x i) is a distance function of the distance between the state space scene and the sampling scene, and may be represented by a euclidean distance, and x represents the sampling scene in the state space scene.
In one illustrative example, the overall equivalence function of the disclosed embodiments is J (x 1,...,xn), the expression of the overall equivalence is:
J(x1,...,xn)＝w1E1+w2E2；
Where w 1 is the weight of the macroscopic equivalent function and w 2 is the weight of the microscopic equivalent function.
The disclosed embodiments determine reliability of a surrogate model for automatic driving based on expert experience or historical data, if the surrogate model reliability is high, the setting w 1 is increased, w 2 is decreased, otherwise, if the surrogate model reliability is low, the setting w 1 is decreased, and w 2 is increased.
In one illustrative example, embodiments of the present disclosure iterate a determined synthetic equivalence function to obtain an optimal sampling scenario, comprising:
a group of sampling scenes x 1,...,xn are determined by utilizing the iterative comprehensive equivalence of a gradient descent method;
Obtaining comprehensive equivalence according to the coverage range of the determined sampling scene x 1,...,xn;
And obtaining the optimal sampling scene by maximizing the comprehensive equivalence function obtained according to the coverage range of the determined sampling scene x 1,...,xn.
It should be noted that, the embodiments of the present disclosure may perform the above iterative operation by using an existing operation method in the related art, which is not limited by the embodiments of the present disclosure.
The embodiment of the disclosure also provides a computer storage medium, in which a computer program is stored, which when executed by a processor, implements the method of test evaluation described above.
The embodiment of the disclosure also provides a terminal, which comprises: a memory and a processor, the memory storing a computer program;
Wherein,
The processor is configured to execute the computer program in the memory;
the computer program, when executed by a processor, implements a method of test evaluation as described above.
The following briefly describes embodiments of the present disclosure by way of application examples, which are merely set forth embodiments of the present disclosure and are not intended to limit the scope of the embodiments of the present disclosure.
Application example
The embodiment of the disclosure provides a test evaluation method, which is an automatic-driving-oriented equivalent test evaluation method, and aims at the situation that the test cost is strictly limited in practical application, and the coverage range of a test scene is measured according to the scene space similarity, so that a small number of test scenes with prominent representativeness are selected, a small number of test results are restored to be equivalent to full test results by utilizing an equivalent test algorithm, the reliability of the test results under the situation that the test cost is strictly ensured, the automatic driving performance index is quantized, and the theoretical accuracy of the test results is ensured while the test efficiency is improved.
The embodiments of the present disclosure note that the theoretical value of the test index (e.g. accident rate) for an autonomous car is mu,Where A represents an event of interest (e.g., a collision accident), p (x) represents the natural probability distribution of test scenario x,/>A test index on scene x of interest event a (e.g., probability of occurrence of event a in scene x);
The test evaluation method in the related art is based on the Monte Carlo sampling principle, and the test index can be evaluated by the following functions:
the method has low test efficiency when the occurrence probability of the event A is extremely low, so that effective test results are difficult to obtain in practical application.
The method used for open road test is a Monte Carlo sampling method, because the accident event occurrence probability of an automatic driving automobile is extremely low, the test method based on Monte Carlo sampling needs extremely many times of test to obtain the estimated value of the accident rate, and the variance of the estimated value is extremely large under the condition of limited test resources (resources such as time, cost and the like), and the estimated result is unreliable.
The test evaluation method of the embodiment of the present disclosure determines the test set size n in advance, where n is generally smaller (e.g., n=10) in the case where the number of tests (the number of tests is limited due to limited test resources such as time, cost, etc.) is smaller than the preset number threshold. Embodiments of the present disclosure extend the Monte Carlo sampling method to a more general case to pre-strategically select sampling scenarios (also referred to as sampling points) x 1,...,xn and calculate an estimate of the test index by a weighted sum
Iteratively selecting a sampling scenario according to a test error that minimizes the sampling scenario (the number of sampling scenarios is less than or equal to a preset number of times threshold, e.g., 10);
e 1 reflects the macroscopic equivalence of the test scene according to the estimated value and the theoretical value of the test index, namely whether the true value of the tested index is accurately equivalent through a group of small-scale tests.
According to the equivalent test evaluation method, the test scene representativeness is measured through the scene coverage, the scene coverage is represented according to the distance from the state space scene to the sampling scene, and the coverage C (x i) of the sampling scene x i is as follows:
Wherein d (x, x i) is a distance function of the distance between the state space scene and the sampling scene, and can be represented by using Euclidean distance, and x represents a point in the state space scene; the weight function of the sampling scene can be calculated according to the coverage range of the sampling scene Further calculate the estimated value/>, of the test index
Since the object to be tested is unknown before the test, and the optimization process needs to estimate the test error, the process is implemented on an automatic driving agent model (Surrogate Model), and when a difference exists between the automatic driving agent model and a real model, the test accuracy may be affected; the disclosed embodiments use microscopic equivalence functions to estimate the error of a proxy model and a real model of autopilot:
the meaning is the difference of the test indexes of each sampling scene x i and other sampling scenes in the coverage range of the sampling scene x i, if the test indexes of the sampling scene and other sampling scenes in the coverage range of the sampling scene are similar, the proxy model of the sampling scene may have stronger reliability; the embodiment of the disclosure eliminates errors caused by inaccurate proxy models to test results to the greatest extent by improving microscopic equivalence.
Finally, the comprehensive equivalence function of the test evaluation method is a combination of a macroscopic equivalence function and a microscopic equivalence function:
J (x 1,...,xn)＝w1E1+w2E2; wherein w 1 is macroscopic equivalent weight and w 2 is microscopic equivalent weight; the embodiment of the disclosure determines the reliability of the automatic driving proxy model based on expert experience or historical data, if the reliability of the proxy model is high, w 1 is set to be increased, w 2 is set to be decreased, otherwise, if the reliability of the proxy model is low, w 1 is set to be decreased, and w 2 is set to be increased.
The embodiment of the disclosure takes the minimized error as the optimized guide, and theoretically quantizes and ensures the accuracy of the test result; compared with the general scene, the method selects the representative scene, improves the testing efficiency of the reliability test of automatic driving, and can ensure the testing reliability under a small number of testing conditions.
The model coverage C (x i) measurement portion of the disclosed embodiments may use a number of different forms, and the form of the distance function d (x, x i) may also be adjusted.
Taking a cut scene of the automatic driving automobile shown in fig. 2 as an example, wherein the tested automatic driving automobile is denoted as an AV, the background automobile is denoted as a BV, and collision with the AV is possible after the BV is changed; cutting yard Jing Dingyi isWherein R represents a relative distance (Range),/>Representing a relative velocity (RANGE RATE); the natural probability distribution p of the cut scene is shown in fig. 3;
According to the test evaluation method of the embodiment of the disclosure, sampling scenes are selected when the test times are 10, and fig. 4 and 5 are effect graphs of sampling scene selection and coverage estimation of the embodiment of the disclosure, and it can be seen from the effect graphs that the test scenes generated by the test evaluation method of the embodiment of the disclosure are distributed in all parts of a scene space, the number of sampling scenes in a general low-risk area is less, and the coverage of a single scene is larger; the method has the advantages that more scenes are sampled in a high-risk area, and the coverage range of a single scene is smaller; therefore, the test evaluation method can fully utilize scene space information, and ensure test reliability when the test cost is limited strictly.
According to the test evaluation method provided by the embodiment of the disclosure, the test evaluation is carried out on the automatic driving automobile, and compared with a Monte Carlo Method (MC) and a random quasi-Monte Carlo method (RQMC) based on uniform sampling, the estimation result of the test index is shown in a table 1; the test results of the three methods on 2 different autopilot models AV-1, AV-2 at a test number of 10 are shown in Table 1.
TABLE 1
By comparison, the estimation error and variance of the equivalent test method are obviously smaller than those of the two reference methods when the test times are strictly limited, so that the test efficiency is improved, and the reliability of a small number of test results is ensured.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Claims (10)
1. A method of test evaluation, comprising:
Determining a coverage area of each sampling scene for a group of sampling scenes;
Determining a weight function of the group of sampling scenes according to the coverage range of each sampling scene;
Determining an estimated value of the test index according to the determined weight function of the set of sampling scenes; obtaining a macroscopic equivalent function according to the theoretical value of the test index and the determined estimated value;
processing errors of the automatic driving agent model and the automatic driving real model to obtain a microcosmic equivalence function;
determining a comprehensive equivalence function of the equivalence test evaluation according to the obtained macroscopic equivalence function and microscopic equivalence function;
Performing iterative operation on the determined comprehensive equivalence function to obtain an optimal sampling scene;
Wherein the set of sampling scenes comprises n sampling scenes, and n is smaller than or equal to a preset numerical value; the microcosmic equivalence function represents the difference of test indexes of the sampling scene x i and other sampling scenes in the coverage range of the sampling scene x i, and x i represents the ith sampling scene in n sampling scenes; the macroscopic equivalence function represents the difference between the estimated value and the theoretical value of the automatic driving integral test index obtained through all sampling scenes.
2. The method of claim 1, wherein said determining an estimate of the test index based on the determined weight function for the set of sampling scenarios comprises:
And calculating the estimated value of the test index in a weighted sum mode according to the weight function.
3. The method according to claim 2, wherein the estimated value of the test index isThe expression of the estimated value is:
Wherein A represents an event of interest, A test index value representing the sampling scenario x i, w (x i) represents the weight function satisfies/>C (x i) represents the coverage of the sampling scene x i.
4. A method according to any one of claims 1 to 3, wherein the macroscopic equivalent function is E 1, the macroscopic equivalent function having the expression:
wherein mu is the theoretical value of the test index, Is an estimate of the test index.
5. The method of claim 4, wherein the microscopic equivalent function is E 2, and the expression of the microscopic equivalent function is:
Where C (x i) represents the coverage of the sampling scene x i, Representing an estimate of the error of the proxy model of autopilot and the real model of autopilot, p (x) representing the probability of occurrence of the scene x in a natural distribution.
6. The method of claim 5, wherein the coverage range C (x i) of the sampling scene x i is expressed as:
where d (x, x i) is a distance function of the state space scene to sample scene distance, x represents the sample scene in the state space scene.
7. The method of claim 5, wherein the integrated equivalence function is J (x 1,...,xn), and wherein the integrated equivalence function has the expression:
J(x1,...,xn)＝w1E1+w2E2；
wherein w 1 is the weight of the macroscopic equivalent function and w 2 is the weight of the microscopic equivalent function.
8. The method of claim 1, wherein iteratively operating the determined synthetic equivalence function to obtain an optimal sampling scenario comprises:
iterating the comprehensive equivalence function by using a gradient descent method to determine a group of sampling scenes x 1,...,xn;
Obtaining a comprehensive equivalence function according to the coverage range of the determined sampling scene x 1,...,xn;
And obtaining the optimal sampling scene by maximizing the comprehensive equivalence function obtained according to the coverage range of the determined sampling scene x 1,...,xn.
9. A computer storage medium having stored therein a computer program which, when executed by a processor, implements the method of test evaluation of any one of claims 1 to 8.
10. A terminal, comprising: a memory and a processor, the memory storing a computer program; wherein,
The processor is configured to execute the computer program in the memory;
the computer program, when executed by the processor, implements a method of test evaluation as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410336952.4A CN118032374A (en) | 2024-03-22 | 2024-03-22 | Test evaluation method, computer storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410336952.4A CN118032374A (en) | 2024-03-22 | 2024-03-22 | Test evaluation method, computer storage medium and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118032374A true CN118032374A (en) | 2024-05-14 |
Family
ID=90993216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410336952.4A Pending CN118032374A (en) | 2024-03-22 | 2024-03-22 | Test evaluation method, computer storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118032374A (en) |
-
2024
- 2024-03-22 CN CN202410336952.4A patent/CN118032374A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160275044A1 (en) | Estimation of Hidden Variance Distribution Parameters | |
| CN106485036B (en) | Method for grading asset securitization asset pool based on survival model | |
| CN113808015B (en) | Spatial interpolation method, device and equipment for meteorological parameters of power transmission line region | |
| CN112463899B (en) | Vehicle track point deviation rectifying method, system, server and storage medium | |
| CN112329272B (en) | Residual life fitting prediction method integrating photovoltaic module degradation and life data | |
| CN107677290B (en) | Testing method and device for precision evaluation of inertial navigation system | |
| CN114359211A (en) | Visibility value detection method, device, equipment and storage medium | |
| CN118032374A (en) | Test evaluation method, computer storage medium and terminal | |
| CN115841049B (en) | Equipment life reliability assessment method and system based on multi-source information fusion | |
| CN111967168A (en) | Optimization design method for accelerated degradation test scheme | |
| CN114048085B (en) | Disk fault analysis method, device, equipment and readable storage medium | |
| CN112561171B (en) | Landslide prediction method, device, equipment and storage medium | |
| CN115755131A (en) | Satellite positioning method, device and medium | |
| CN114758080A (en) | Sea surface salinity gridding inversion method and device | |
| CN113640760A (en) | Radar discovery probability evaluation method and equipment based on air situation data | |
| US20170220711A1 (en) | Flow rate prediction device, mixing ratio estimation device, method, and computer-readable recording medium | |
| CN116009516A (en) | Method and device for realizing self-adaptive acceleration test | |
| CN113701768A (en) | Path determination method and device and electronic equipment | |
| CN116680517B (en) | Method and device for determining failure probability in automatic driving simulation test | |
| CN116360387B (en) | Fault positioning method integrating Bayesian network and performance-fault relation map | |
| CN116088476A (en) | Method and device for realizing self-adaptive acceleration test evaluation | |
| CN116663939B (en) | Unmanned vehicle path planning scene and task complexity evaluation method and system | |
| KR20240084082A (en) | Apparatus and method for predicting remaining useful life based on adaptive inverse power law | |
| CN113052582B (en) | Method, device, equipment and computer storage medium for checking bill | |
| CN107218964B (en) | Method for judging capacity character of test subsample |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |