CN115114167A - Method and device for evaluating functions of automatic driving system and storage medium - Google Patents
Method and device for evaluating functions of automatic driving system and storage medium Download PDFInfo
- Publication number
- CN115114167A CN115114167A CN202210811303.6A CN202210811303A CN115114167A CN 115114167 A CN115114167 A CN 115114167A CN 202210811303 A CN202210811303 A CN 202210811303A CN 115114167 A CN115114167 A CN 115114167A
- Authority
- CN
- China
- Prior art keywords
- scene
- test
- evaluation
- current test
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Educational Administration (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Computer Hardware Design (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses an evaluation method, an evaluation device and a storage medium for functions of an automatic driving system, wherein the method comprises the following steps: acquiring scene element labels of functions to be tested of the vehicle, and acquiring a plurality of test scene cases according to the scene element labels; performing data analysis on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement and determine scene element parameter items in the test scene case adapted to the current test requirement; acquiring an evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement; and executing a test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element. Therefore, the functions to be tested of the vehicle are accurately evaluated.
Description
Technical Field
The invention relates to the technical field of vehicle function testing, in particular to an evaluation method of an automatic driving system function, a computer readable storage medium and an evaluation device of the automatic driving system function.
Background
At present, simulation test evaluation of functions of an automatic driving system is mostly based on standards or developed in a subjective evaluation form, however, a problem of the related art is that in the process of performing simulation test evaluation of functions of the automatic driving system, artificial setting of standards or subjective evaluation indexes is usually involved, so that simulation test evaluation in a natural driving scene cannot be accurately realized.
Disclosure of Invention
The present invention is directed to solving, at least in part, one of the technical problems in the related art. Therefore, a first objective of the present invention is to provide an evaluation method for functions of an automatic driving system, which can execute a test scenario case adapted to a current test requirement, and evaluate a to-be-tested function of a vehicle according to an evaluation index of each scenario element, thereby accurately implementing evaluation of functions of an automatic driving system in a natural driving scenario.
A second object of the invention is to propose a computer-readable storage medium.
A third object of the present invention is to provide an evaluation device for the function of an automatic driving system.
In order to achieve the above object, an evaluation method for a function of an automatic driving system according to an embodiment of a first aspect of the present invention includes the steps of: the method comprises the steps of obtaining scene element labels of functions to be tested of a vehicle, and obtaining a plurality of test scene cases according to the scene element labels; performing data analysis on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement and determine scene element parameter items in the test scene case adapted to the current test requirement; acquiring an evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement; and executing a test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element.
According to the evaluation method of the automatic driving system function provided by the embodiment of the invention, the scene element label of the function to be tested of the vehicle is obtained, and a plurality of test scene cases are obtained according to the scene element label, and further, performing data analysis on a plurality of scene elements in each test scene case according to the current test requirement, so as to determine the test scenario case adapted to the current test requirement, and determine the scenario element parameter item in the test scenario case adapted to the current test requirement, then, obtaining the evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement, and executing a test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element. Therefore, the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
In addition, the evaluation method of the automatic driving system function according to the above embodiment of the present invention may further have the following additional technical features:
according to an embodiment of the present invention, the determining a test scenario case adapted to the current test requirement includes: and obtaining scene elements corresponding to the current test requirements, and screening the test scene use cases according to the scene elements corresponding to the current test requirements to determine the test scene use cases adapted to the current test requirements.
According to one embodiment of the invention, the scene element labels of the functions to be tested of the vehicle comprise a scene static parameter label and a scene dynamic parameter label.
According to one embodiment of the invention, the scene static parameter labels comprise one or more of lane type, number of lanes in the same direction, lane where the test vehicle is located, lane width, road surface gradient, road surface adhesion condition, test vehicle type and weather state, and the scene static parameter labels comprise one or more of initial speed of the test vehicle, initial distance from lane center line of the test vehicle, lateral deviation speed of the test vehicle and deviation direction of the test vehicle.
According to an embodiment of the present invention, the obtaining an evaluation index of each scene element in a test scene case adapted to the current test requirement includes: and carrying out sectional statistics on the dynamic parameters in each scene element in the test scene case corresponding to the current test requirement, and acquiring the evaluation index of the dynamic parameters according to a 3 sigma principle.
According to an embodiment of the present invention, the evaluation index of the dynamic parameter includes an optimal value evaluation index and a passing value evaluation index of the dynamic parameter.
According to an embodiment of the present invention, the executing a test scenario case adapted to the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scenario element includes: repeatedly executing the test scene case adapted to the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element so as to obtain the evaluation corresponding to the test scene case adapted to the current test requirement; and carrying out weighted average on the evaluation of each test scene case which is matched with the current test requirement so as to obtain the final evaluation of the vehicle function to be tested.
According to one embodiment of the invention, the final evaluation of the vehicle function to be tested is obtained according to the following formula: score final ＝∑ i Score i ×Weight i Wherein, Score final Score for the final evaluation of the vehicle function to be tested i Score for the ith test, Weight i Is the ith test weight.
To achieve the above object, a computer-readable storage medium according to an embodiment of a second aspect of the present invention is provided, on which an evaluation program of an autopilot system function is stored, which when executed by a processor implements the method for evaluating an autopilot system function according to the embodiment of the first aspect.
According to the computer-readable storage medium provided by the embodiment of the invention, the test scene case adaptive to the current test requirement can be executed, and the to-be-tested function of the vehicle is evaluated according to the evaluation index of each scene element, so that the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
In order to achieve the above object, an evaluation device for evaluating a function of an automatic driving system according to a third aspect of the present invention includes: the system comprises a test case acquisition module, a test case analysis module and a test case analysis module, wherein the test case acquisition module is used for acquiring scene element labels of functions to be tested of a vehicle and acquiring a plurality of test scene cases according to the scene element labels; the analysis module is used for carrying out data analysis on a plurality of scene elements in each test scene case according to the current test requirement so as to determine the test scene case adaptive to the current test requirement and determine scene element parameter items in the test scene case adaptive to the current test requirement; the evaluation index acquisition module is used for acquiring the evaluation index of each scene element in the test scene case which is adaptive to the current test requirement according to the test scene case which is adaptive to the current test requirement and the scene element parameter item in the test scene case which is adaptive to the current test requirement; and the evaluation module is used for executing the test scene case matched with the current test requirement and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element.
According to the evaluation device of the automatic driving system function provided by the embodiment of the invention, the scene element label of the vehicle function to be tested is obtained through the test case obtaining module, a plurality of test scene cases are obtained according to the scene element label, furthermore, the data analysis is carried out on a plurality of scene elements in each test scene case through the analysis module according to the current test requirement so as to determine the test scene case matched with the current test requirement and determine the scene element parameter item in the test scene case matched with the current test requirement, then, the evaluation index of each scene element in the test scene case matched with the current test requirement is obtained through the evaluation index obtaining module according to the test scene case matched with the current test requirement and the scene element parameter item in the test scene case matched with the current test requirement, and executing the test scene case adapted to the current test requirement through the evaluation module, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element. Therefore, the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a schematic flow diagram of a method for evaluating the functionality of an autopilot system according to an embodiment of the invention;
FIG. 2 is a flow diagram of a method for evaluating the functionality of an autopilot system according to one embodiment of the invention;
FIG. 3 is a schematic flow diagram of a method for evaluating the functionality of an autopilot system according to another embodiment of the invention;
fig. 4 is a block schematic diagram of an evaluation device of an autopilot system function according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
An evaluation method of an autonomous driving system function, a computer-readable storage medium, and an evaluation apparatus of an autonomous driving system function according to an embodiment of the present invention are described below with reference to the accompanying drawings.
Fig. 1 is a flowchart illustrating an evaluation method of an autopilot system function according to an embodiment of the invention.
As shown in fig. 1, the method for evaluating the function of the automatic driving system includes the following steps:
s101, obtaining scene element labels of functions to be tested of the vehicle, and obtaining a plurality of test scene cases according to the scene element labels.
It is understood that, in the embodiment of the present invention, the scene element labels of the vehicle functions to be tested are scene elements in a 6-layer structure model defined by PEGASUS, wherein different vehicle functions to be tested may correspond to different scene element labels.
Optionally, in the embodiment of the present invention, a scene element tag according to a function to be tested of a vehicle may be obtained, and a plurality of test scene cases may be obtained from a natural driving scene library.
S102, performing data analysis on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement and determine scene element parameter items in the test scene case adapted to the current test requirement.
It is understood that, since the test requirements of the autonomous driving system function to be evaluated each time may be the same or different, for example, assuming that the function to be tested of the vehicle is an LKA (Lane Keeping Assist) function in the autonomous driving system function, the test requirements based on the LKA function may include straight centering control, curve centering control, straight Lane deviation suppression, and curve Lane deviation suppression, wherein the scene elements required by the test requirements of the different LKA functions are different, for example, the scene elements required to be set by the straight centering control only include an initial speed and an initial deviation Lane center line distance, and the scene elements required to be set by the straight Lane deviation suppression also include an initial speed, a lateral deviation speed, and a deviation direction.
Therefore, in the embodiment of the present invention, data analysis may be performed on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement, and a scene element parameter item in the test scene case adapted to the current test requirement may be determined by an expert scoring method, so as to subsequently obtain an evaluation index of the scene element parameter item in the test scene case adapted to the current test requirement.
S103, obtaining the evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement.
It can be understood that, in the embodiment of the present invention, the evaluation index of each scene element in the test scenario case adapted to the current test requirement may be obtained according to the test scenario case adapted to the current test requirement and the scenario element parameter item in the test scenario case adapted to the current test requirement, so as to evaluate the to-be-tested function of the vehicle according to the evaluation index of each scene element.
And S104, executing a test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element.
It should be understood that, in some embodiments of the present invention, a scene element label of a function to be tested of a vehicle is first obtained, a plurality of test scene cases are obtained according to the scene element label, so as to implement primary screening of the test scene cases, and then data analysis is performed on a plurality of scene elements in each test scene case according to a current test requirement, so as to determine the test scene case adapted to the current test requirement, so as to implement secondary screening of the test scene cases, and determine scene element parameter items in the test scene cases adapted to the current test requirement, so as to determine an evaluation object of the test scene cases, and then an evaluation index of each scene element in the test cases adapted to the current test requirement is obtained according to the test scene cases adapted to the current test requirement, and the scene element parameter items in the test scene cases adapted to the current test requirement, therefore, the evaluation index of each evaluation object in the test scene case is determined, the test scene case adaptive to the current test requirement is executed, and the to-be-tested function of the vehicle is evaluated according to the evaluation index of each scene element, so that the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
Further, determining a test scenario case adapted to the current test requirement includes: and obtaining scene elements corresponding to the current test requirements, and screening test scene cases according to the scene elements corresponding to the current test requirements to determine the test scene cases adapted to the current test requirements.
Specifically, because the scene elements corresponding to different test requirements are different, in some embodiments of the present invention, the test scene case may be further screened according to the scene element corresponding to the current test requirement, so as to screen out the test scene case adapted to the current test requirement, thereby implementing secondary screening of the test scene case, so as to implement, by executing the test scene case adapted to the current test requirement, the function evaluation of the automatic driving system in the natural driving scene accurately.
Further, the scene element labels of the functions to be tested of the vehicle comprise a scene static parameter label and a scene dynamic parameter label.
Specifically, in some embodiments of the present invention, a scene static parameter tag and a scene dynamic parameter tag of a function to be tested of a vehicle may be obtained, and a plurality of test scene cases may be obtained from a natural driving scene library according to the scene static parameter tag and the scene dynamic parameter tag, so as to implement initial screening of the test scene cases, so as to accurately implement function evaluation of an automatic driving system in a natural driving scene.
Further, the static scene parameter labels comprise one or more of lane type, number of lanes in the same direction, lane width of the test vehicle, road gradient, road adhesion condition, type of the test vehicle and weather state, and the static scene parameter labels comprise one or more of initial speed of the test vehicle, initial departure distance of the test vehicle from the center line of the lane, lateral departure speed of the test vehicle and departure direction of the test vehicle.
Specifically, in some embodiments of the present invention, a plurality of test scenario cases may be obtained from the natural driving scenario library according to one or more of lane types, the number of lanes in the same direction, lanes in which test vehicles are located, lane widths, road slopes, road adhesion conditions, test vehicle types, and weather conditions, and one or more of an initial speed of a test vehicle, an initial lane center line distance of a test vehicle, a lateral deviation speed of a test vehicle, and a deviation direction of a test vehicle, so as to implement initial screening of the test scenario cases, so as to accurately implement functional evaluation of the automatic driving system in a natural driving scenario.
Further, obtaining an evaluation index of each scene element in the test scene case adapted to the current test requirement includes: and carrying out sectional statistics on the dynamic parameters in each scene element in the test scene case corresponding to the current test requirement, and obtaining the evaluation index of the dynamic parameters according to the 3 sigma principle.
Specifically, in some embodiments of the present invention, a dynamic parameter in each scene element in a test scene case corresponding to a current test requirement may be subjected to segmented statistics, so as to determine a probability distribution state of each dynamic parameter, and then an evaluation index of the dynamic parameter is obtained according to a 3 sigma principle, so as to obtain an evaluation index of each scene element in the test scene case adapted to the current test requirement, so as to execute the test scene case adapted to the current test requirement, and evaluate a function to be tested of a vehicle according to the evaluation index of each scene element, thereby accurately implementing an evaluation of an automatic driving system function in a natural driving scene.
Further, the evaluation indexes of the dynamic parameters comprise an optimal value evaluation index and a dynamic parameter and qualification value evaluation index of the dynamic parameters.
Optionally, the optimal value evaluation index of the dynamic parameter, the dynamic parameter passing value evaluation index may be fixed numerical values, or may be intervals.
Specifically, in some embodiments of the present invention, the dynamic parameters in each scene element in the test scene case corresponding to the current test requirement may be subjected to segmented statistics, and the optimal value evaluation index and the dynamic parameter and the qualification evaluation index of the dynamic parameters are obtained according to the 3 sigma principle, so as to execute the test scene case adapted to the current test requirement, and the function to be tested of the vehicle is evaluated according to the optimal value evaluation index and the dynamic parameter and the qualification evaluation index of the dynamic parameters, thereby accurately implementing the function evaluation of the automatic driving system in the natural driving scene.
Further, as shown in fig. 2, executing a test scenario case adapted to the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scenario element includes:
s201, repeatedly executing the test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element to obtain the evaluation corresponding to the test scene case matched with the current test requirement.
It can be understood that, in the embodiment of the present invention, the test scenario case adapted to the current test requirement may be executed for multiple times, and the to-be-tested function of the vehicle may be evaluated according to the evaluation index of each scenario element, so as to obtain the evaluation corresponding to the test scenario case adapted to the current test requirement, which is executed each time, so as to perform weighted average on the evaluation of the test scenario case adapted to the current test requirement, which is executed each time, subsequently, so as to obtain the final evaluation of the to-be-tested function of the vehicle.
S202, carrying out weighted average on the evaluation of the test scene case which is adapted to the current test requirement every time so as to obtain the final evaluation of the vehicle function to be tested.
It should be understood that, because the test scenario case adapted to the current test requirement is executed each time, and the evaluation results of evaluating the to-be-tested function of the vehicle according to the evaluation index of each scenario element may not be the same, in order to more accurately realize the evaluation of the function of the automatic driving system in the natural driving scenario, in some embodiments of the present invention, the evaluation corresponding to the test scenario case adapted to the current test requirement executed each time may also be weighted and averaged to obtain the final evaluation of the to-be-tested function of the vehicle.
Further, the final evaluation of the vehicle function to be tested is obtained according to the following formula:
Score final ＝∑ i Score i ×Weight i ，
wherein, Score final For the final evaluation of the function to be tested of the vehicle, Score i For the ith test score, Weight i Is the ith test weight.
It should be noted that, if the test scenario case with the adaptive current test requirement is executed and the number of times of evaluating the to-be-tested vehicle function according to the evaluation index of each scenario element is smaller, the test efficiency of the automatic driving system function is higher, but the test accuracy is poor, otherwise, if the test scenario case with the adaptive current test requirement is executed and the number of times of evaluating the to-be-tested vehicle function according to the evaluation index of each scenario element is larger, the final evaluation result of the automatic driving system function is more accurate, but the test efficiency is lower, for this reason, the number of times of executing the test scenario case with the adaptive current test requirement can be set according to different actual requirements, which is not limited herein.
Referring to fig. 3 and the specific embodiment of the present invention, the specific steps executed in the method for evaluating the function of the automatic driving system according to the embodiment of the present invention will be described, and as shown in fig. 3, after the algorithm corresponding to the method for evaluating the function of the automatic driving system is executed, step S1 is executed.
And S1, specifying the function to be tested of the vehicle.
And S2, acquiring scene element labels corresponding to the to-be-tested functions of the vehicle.
And S3, acquiring a plurality of test scenario cases according to the scenario element labels.
And S4, performing data analysis on the scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement.
And S5, determining the scene element parameter item in the test scene case which is adapted to the current test requirement.
And S6, obtaining the evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement.
And S7, repeatedly executing the test scene case adapted to the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element to obtain the evaluation corresponding to the test scene case adapted to the current test requirement.
And S8, carrying out weighted average on the evaluation corresponding to the test scene case which is adapted to the current test requirement in each execution, so as to obtain the final evaluation of the vehicle function to be tested.
In summary, according to the evaluation method for the functions of the automatic driving system provided by the embodiment of the present invention, the scene element label of the function to be tested of the vehicle is obtained, and a plurality of test scene cases are obtained according to the scene element label, and further, data analysis is performed on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement, and determine the scene element parameter item in the test scene case adapted to the current test requirement, and then, according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement, the evaluation index of each scene element in the test scene case adapted to the current test requirement is obtained, and the test scene case adapted to the current test requirement is executed, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element. Therefore, the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
Based on the foregoing evaluation method of the autopilot system function according to the embodiment of the invention, an embodiment of the invention proposes a computer-readable storage medium having stored thereon an evaluation program of the autopilot system function, which when executed by a processor implements the evaluation method of the autopilot system function according to the foregoing embodiment of the invention.
It should be noted that, when executing the evaluation program of the autopilot system function, the computer-readable storage medium according to the embodiment of the present invention can implement the specific implementation manner corresponding to the evaluation method of the autopilot system function according to the foregoing embodiment of the present invention, and details are not repeated herein.
In summary, the computer-readable storage medium provided by the embodiment of the invention can execute the test scenario case adapted to the current test requirement, and evaluate the to-be-tested function of the vehicle according to the evaluation index of each scenario element, thereby accurately implementing the evaluation of the function of the automatic driving system in the natural driving scenario.
Fig. 4 is a block schematic diagram of an evaluation device of an autopilot system function according to an embodiment of the invention.
As shown in fig. 4, the evaluation device 100 for the automatic driving system function includes: the test case analysis system comprises a test case acquisition module 10, an analysis module 20, an evaluation index acquisition module 30 and an evaluation module 40.
Specifically, the test case obtaining module 10 is configured to obtain a scene element label of a function to be tested of the vehicle, and obtain a plurality of test scene cases according to the scene element label; the analysis module 20 is configured to perform data analysis on a plurality of scene elements in each test scene case according to the current test requirement, to determine a test scene case adapted to the current test requirement, and to determine scene element parameter items in the test scene case adapted to the current test requirement; the evaluation index obtaining module 30 is configured to obtain an evaluation index of each scene element in the test scenario case adapted to the current test requirement according to the test scenario case adapted to the current test requirement and the scene element parameter item in the test scenario case adapted to the current test requirement; the evaluation module 40 is configured to execute a test scenario case adapted to the current test requirement, and evaluate the to-be-tested function of the vehicle according to the evaluation index of each scenario element.
Further, the analysis module 20 is further configured to obtain a scene element corresponding to the current test requirement, and screen the test scene case according to the scene element corresponding to the current test requirement, so as to determine the test scene case adapted to the current test requirement.
Further, the scene element labels of the functions to be tested of the vehicle comprise a scene static parameter label and a scene dynamic parameter label.
Further, the static scene parameter labels comprise one or more of lane type, number of lanes in the same direction, lane width of the test vehicle, road gradient, road adhesion condition, type of the test vehicle and weather state, and the static scene parameter labels comprise one or more of initial speed of the test vehicle, initial departure distance of the test vehicle from the center line of the lane, lateral departure speed of the test vehicle and departure direction of the test vehicle.
Further, the evaluation index obtaining module 30 is further configured to perform segmented statistics on the dynamic parameters in each scene element in the test scene case corresponding to the current test requirement, and obtain the evaluation index of the dynamic parameters according to a 3 sigma principle.
Further, the evaluation indexes of the dynamic parameters comprise an optimal value evaluation index and a dynamic parameter and qualification value evaluation index of the dynamic parameters.
Further, the evaluation module 40 is further configured to repeatedly execute the test scenario case adapted to the current test requirement, and evaluate the to-be-tested function of the vehicle according to the evaluation index of each scenario element, so as to obtain the evaluation corresponding to each execution of the test scenario case adapted to the current test requirement; and carrying out weighted average on the evaluation of each test scene case which is matched with the current test requirement so as to obtain the final evaluation of the vehicle function to be tested.
Further, the evaluation module 40 is further configured to obtain a final evaluation of the function to be tested of the vehicle according to the following formula: score final ＝∑ i Score i ×Weight i Wherein, Score final For the final evaluation of the function to be tested of the vehicle, Score i For the ith test score, Weight i Is the ith test weight.
It should be noted that, for a specific implementation of the evaluation apparatus 100 for an autopilot system function according to the embodiment of the present invention, reference may be made to the specific implementation of the evaluation method for an autopilot system function according to the foregoing embodiment of the present invention, and details are not described herein again.
In summary, according to the evaluation apparatus for the functions of the automatic driving system provided by the embodiment of the present invention, the test case obtaining module obtains the scene element label of the function to be tested of the vehicle, and obtains a plurality of test scene cases according to the scene element label, further, the analysis module performs data analysis on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement, and determine the scene element parameter item in the test scene case adapted to the current test requirement, and then, the evaluation index obtaining module obtains the evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement, and the scene element parameter item in the test scene case adapted to the current test requirement, and executing the test scene case adapted to the current test requirement through the evaluation module, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element. Therefore, the function evaluation of the automatic driving system in the natural driving scene is accurately realized.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and are therefore not to be considered limiting of the invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature "under," "beneath," and "under" a second feature may be directly under or obliquely under the second feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A method for evaluating the function of an automatic driving system, the method comprising the steps of:
the method comprises the steps of obtaining scene element labels of functions to be tested of a vehicle, and obtaining a plurality of test scene cases according to the scene element labels;
performing data analysis on a plurality of scene elements in each test scene case according to the current test requirement to determine the test scene case adapted to the current test requirement and determine scene element parameter items in the test scene case adapted to the current test requirement;
acquiring an evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement;
and executing a test scene case matched with the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element.
2. The method of evaluating the functionality of an autopilot system according to claim 1, wherein the determining a test scenario case adapted to the current test requirements comprises:
and obtaining scene elements corresponding to the current test requirements, and screening the test scene use cases according to the scene elements corresponding to the current test requirements to determine the test scene use cases adapted to the current test requirements.
3. The method for evaluating an autopilot system function according to one of claims 1-2, characterized in that the scene element labels of the vehicle functions to be tested comprise a scene static parameter label and a scene dynamic parameter label.
4. The method of evaluating an autopilot system function of claim 3 wherein the scene static parameter labels include one or more of a lane type, a number of co-directional lanes, a lane in which the test vehicle is located, a lane width, a road grade, a road adhesion condition, a test vehicle type, and a weather condition, and wherein the scene static parameter labels include one or more of a test vehicle initial speed, a test vehicle initial departure from a lane centerline distance, a test vehicle lateral departure speed, and a test vehicle departure direction.
5. The method for evaluating the function of the automatic driving system according to claim 3, wherein the obtaining the evaluation index of each scene element in the test scene case adapted to the current test requirement comprises:
and carrying out sectional statistics on the dynamic parameters in each scene element in the test scene case corresponding to the current test requirement, and acquiring the evaluation index of the dynamic parameters according to a 3 sigma principle.
6. The method of claim 5, wherein the evaluation indicators of the dynamic parameters include an optimal evaluation indicator of the dynamic parameters and a passing evaluation indicator of the dynamic parameters.
7. The method for evaluating the function of the automatic driving system according to claim 1, wherein the executing a test scenario case adapted to the current test requirement and evaluating the function to be tested of the vehicle according to the evaluation index of each scenario element comprises:
repeatedly executing the test scene case adapted to the current test requirement, and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element so as to obtain the evaluation corresponding to the test scene case adapted to the current test requirement;
and carrying out weighted average on the evaluation of the test scene case which is matched with the current test requirement and is executed each time so as to obtain the final evaluation of the to-be-tested function of the vehicle.
8. The method of evaluating an autopilot system function according to claim 7, characterized in that the final evaluation of the vehicle function to be tested is obtained according to the following formula:
Score final ＝∑ i Score i ×Weight i ，
wherein, Score final Score for the final evaluation of the vehicle function to be tested i For the ith test score, Weight i Is the ith test weight.
9. A computer-readable storage medium, characterized in that an evaluation program of an automatic driving system function is stored thereon, which when executed by a processor implements the evaluation method of an automatic driving system function according to any one of claims 1 to 8.
10. An apparatus for evaluating a function of an automatic driving system, the apparatus comprising:
the system comprises a test case acquisition module, a test case analysis module and a test case analysis module, wherein the test case acquisition module is used for acquiring scene element labels of functions to be tested of a vehicle and acquiring a plurality of test scene cases according to the scene element labels;
the analysis module is used for carrying out data analysis on a plurality of scene elements in each test scene case according to the current test requirement so as to determine the test scene case adaptive to the current test requirement and determine scene element parameter items in the test scene case adaptive to the current test requirement;
the evaluation index acquisition module is used for acquiring the evaluation index of each scene element in the test scene case adapted to the current test requirement according to the test scene case adapted to the current test requirement and the scene element parameter item in the test scene case adapted to the current test requirement;
and the evaluation module is used for executing the test scene case matched with the current test requirement and evaluating the to-be-tested function of the vehicle according to the evaluation index of each scene element.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210811303.6A CN115114167A (en) | 2022-07-11 | 2022-07-11 | Method and device for evaluating functions of automatic driving system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210811303.6A CN115114167A (en) | 2022-07-11 | 2022-07-11 | Method and device for evaluating functions of automatic driving system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115114167A true CN115114167A (en) | 2022-09-27 |
Family
ID=83332162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210811303.6A Pending CN115114167A (en) | 2022-07-11 | 2022-07-11 | Method and device for evaluating functions of automatic driving system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115114167A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116358902A (en) * | 2023-06-02 | 2023-06-30 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
-
2022
- 2022-07-11 CN CN202210811303.6A patent/CN115114167A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116358902A (en) * | 2023-06-02 | 2023-06-30 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
| CN116358902B (en) * | 2023-06-02 | 2023-08-22 | 中国第一汽车股份有限公司 | Vehicle function testing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108154508B (en) | Method, apparatus, storage medium and the terminal device of product defects detection positioning | |
| CN113091866B (en) | Method and device for measuring loading quality of automobile in real time | |
| JP5313250B2 (en) | Battery long-term characteristic prediction system and method | |
| CN115114167A (en) | Method and device for evaluating functions of automatic driving system and storage medium | |
| CN110174110B (en) | Map corresponding method and device, electronic equipment and computer readable medium | |
| JP2010538246A (en) | Battery long-term characteristic prediction system and method | |
| CN111121939B (en) | High-precision vehicle-mounted area weighing method | |
| CN111157092A (en) | Vehicle-mounted weighing automatic calibration method and computer readable storage medium | |
| CN115790804B (en) | Dynamic truck scale state monitoring method, medium, equipment and device | |
| CN111708856A (en) | Crowdsourcing data segmentation fusion method of lane line based on reinforcement learning | |
| CN112183381A (en) | Method and device for detecting driving area of vehicle | |
| US11226373B2 (en) | Method for determining a state noise covariance matrix for adjusting an observer of the state of charge of a battery and corresponding device | |
| CN117576919B (en) | Vehicle overload recognition system and method and electronic equipment | |
| CN112070155A (en) | Time series data labeling method and device | |
| CN112183382A (en) | Unmanned traffic light detection and classification method and device | |
| US20240183753A1 (en) | Method and vehicle system for determining the state of the components of a chassis | |
| CN113160128B (en) | Defect detection method for LED and storage medium | |
| CN114235448A (en) | Rail vehicle bogie wheel fatigue damage assessment method and system | |
| CN115469635A (en) | Verification method, device and equipment for safety of automatic driving simulation test function | |
| CN111582308B (en) | Method, device and storage medium for calculating reliability of OPC modeling data | |
| CN114495509A (en) | Method for monitoring tunnel running state based on deep neural network | |
| CN116086498B (en) | Slope sensor zeroing method and device, computer equipment and storage medium | |
| CN116503830B (en) | Method and device for testing target detection algorithm and server | |
| CN117150275B (en) | Machine learning model construction method, battery health degree prediction method and device | |
| US20220402504A1 (en) | Methods and Systems for Generating Ground Truth Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |