CN115129027A - Automatic evaluation method and device for intelligent driving - Google Patents
Automatic evaluation method and device for intelligent driving Download PDFInfo
- Publication number
- CN115129027A CN115129027A CN202210699670.1A CN202210699670A CN115129027A CN 115129027 A CN115129027 A CN 115129027A CN 202210699670 A CN202210699670 A CN 202210699670A CN 115129027 A CN115129027 A CN 115129027A
- Authority
- CN
- China
- Prior art keywords
- test case
- parameters
- vehicle state
- simulation
- inputting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B23/00—Testing or monitoring of control systems or parts thereof
- G05B23/02—Electric testing or monitoring
- G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
- G05B23/0208—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the configuration of the monitoring system
- G05B23/0213—Modular or universal configuration of the monitoring system, e.g. monitoring system having modules that may be combined to build monitoring program; monitoring system that can be applied to legacy systems; adaptable monitoring system; using different communication protocols
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/24—Pc safety
- G05B2219/24065—Real time diagnostics
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention provides an automatic evaluation method and device for intelligent driving, wherein the method comprises the following steps: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters; and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard. Automatic assessment and intelligent driving test are achieved. The independent operation and effective cooperation of the simulation test system and the evaluation work are realized.
Description
Technical Field
The invention relates to the field of intelligent driving tests, in particular to an automatic evaluation method and device for intelligent driving.
Background
The intelligent driving vehicle sends high-precision relevant data such as longitude and latitude, height, destination and the like to the control center through the communication system. The control center processes all vehicle state information, destination information and the like, plans an optimal route for all vehicles, and transmits the control information of the vehicles to all vehicles in real time, so that intelligent driving in the whole system is realized. However, driving tests for smart vehicles have exponentially increased verification relative to conventional vehicles, presenting significant challenges in time, labor, and testing costs.
In the prior art of intelligent driving test, quantitative evaluation methods of different dimensions mainly depend on the experience of a test engineer, and test evaluation of a huge simulation scene library is almost completed by the test engineer. Therefore, the prior art finds the initial positioning and specific analysis of the test problems, depending on the individual investigation by the test engineer.
In addition, in the prior art, the output of the intelligent driving simulation test and the evaluation result is basically performed in two steps, the test result evaluation mainly depends on the output of the evaluation result by an engineer after the test is finished, linkage and real-time performance are not provided, and the simulation test system and the evaluation work operate independently, so that the overall linkage and real-time performance are poor, and effective cooperation and parallelism cannot be realized.
Disclosure of Invention
The invention provides an automatic evaluation method and device for intelligent driving.
In a first aspect, the present invention provides an automated assessment method for smart driving, comprising: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters; and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
Further, the obtaining a test case, inputting the test case into an intelligent driving controller, and determining control driving parameters of the test case in a simulation scene includes: acquiring a test case, determining parameters of a simulation scene according to the test case, inputting the parameters of the simulation scene and the test case into an intelligent driving controller, and determining control driving parameters of the test case in the simulation scene.
Further, the obtaining of the test case, inputting the test case into the intelligent driving controller, determining the control driving parameters of the test case in the simulation scene, and inputting the control driving parameters into the vehicle model to obtain the vehicle state parameters includes: acquiring a test case, inputting the test case into an intelligent driving controller, determining the control driving parameters of the test case at the current moment under a simulation scene, and inputting the control driving parameters at the current moment into a vehicle model to obtain the vehicle state parameters at the current moment; inputting the vehicle state parameters at the current moment into an intelligent driving controller, determining the control driving parameters corresponding to the vehicle state parameters at the current moment at the next moment under a simulation scene, and inputting the control driving parameters at the next moment into the vehicle model to obtain the vehicle state parameters at the next moment; and circulating the steps until the test case finishes the test.
Further, the determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters includes: determining whether the simulation system is normal or not according to the test case, the control driving parameters at the current moment and the vehicle state parameters at the current moment; and/or determining whether the simulation system is normal according to the test case, the control driving parameters at the next moment and the vehicle state parameters at the next moment.
Further, the determining whether the simulation system is normal according to the test case, the control driving parameter and the vehicle state parameter includes: and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal.
Further, the observer includes a system state observer, a scene observer, and a communication diagnostor; and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal, wherein the method comprises the following steps: inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the state of a simulation system is normal; inputting the parameters of the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal; and inputting signals of a preset communication protocol and the vehicle state parameters into a communication diagnostor to determine whether the communication is normal or not.
Further, the inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the simulation system state is normal includes: determining a predicted vehicle state parameter at the next moment according to the vehicle state parameter at the current moment and the control driving parameter at the current moment; and acquiring the vehicle state parameter at the next moment, determining the deviation value between the vehicle state parameter at the next moment and the predicted vehicle state parameter at the next moment, and if the deviation value does not meet the preset range, judging that the state of the simulation system is abnormal.
Further, the test case comprises a corresponding design operation domain, the simulation scenario also comprises a corresponding design operation domain, and the design operation domain is used for representing key parameters of a driving scenario simulated by the test case; and inputting the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal, wherein the method comprises the following steps: and inputting the design operation domain corresponding to the test case and the design operation domain corresponding to the simulation scene into a scene observer to determine whether the simulation scene is normal.
Further, the inputting a signal of a preset communication protocol and the vehicle state parameter into the communication diagnostor, and determining whether the communication is normal, comprises: and the communication diagnostor determines the transmission cycle, the signal name and the verification information of the signal according to the received signal of the vehicle state parameter, and checks whether the transmission cycle, the signal name and the verification information of the signal are normal or not through a preset communication protocol.
Further, the evaluation criteria include at least one of: comfort, reliability, fuel economy, safety.
In a second aspect, the present invention also provides an automated evaluation apparatus for smart driving, comprising: the simulation module is used for acquiring a test case, inputting the test case into the intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; the simulation system evaluation module is used for determining whether the simulation system is normal or not according to the test case, the control driving parameters and the vehicle state parameters; and the intelligent driving controller evaluation module is used for waiting for the test case to finish the test if the test case is normal, obtaining a test result and evaluating the test result according to a preset evaluation standard.
In a third aspect, the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the automated evaluation method for smart driving as described in any one of the above.
In a fourth aspect, the invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the automated assessment method for smart driving as described in any of the above.
In a fifth aspect, the present invention also provides a computer program product comprising a computer program which, when being executed by a processor, carries out the steps of the automated assessment method for intelligent driving as described in any of the above.
According to the automatic evaluation method and device for intelligent driving, the test case is input into the intelligent driving controller, the control driving parameters of the test case in the simulation scene are determined, the control driving parameters are input into the vehicle model, the vehicle state parameters are obtained, the test period of the simulation test work of the intelligent driving huge test scene library is shortened, and the test efficiency is improved. And determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters, if so, waiting for the test case to finish the test to obtain a test result, evaluating the test result according to a preset evaluation standard, so that the simulation test and the evaluation can be effectively coordinated and parallel, the overall linkage and the real-time performance of the simulation test and the evaluation are improved, and the simulation system and the test result are respectively evaluated to realize the primary classification and positioning of problems which may appear in the driving test process.
Drawings
In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow diagram of some embodiments of an automated assessment method for intelligent driving provided in accordance with the present invention;
FIG. 2 is a schematic diagram of a framework for an automated assessment method for intelligent driving;
FIG. 3 is a schematic block diagram of some embodiments of an automated evaluation device for intelligent driving provided in accordance with the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in accordance with the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present invention are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in the present invention are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that reference to "one or more" unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present invention are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
Referring to fig. 1, fig. 1 is a flow chart illustrating an automated evaluation method for intelligent driving according to some embodiments of the present invention. As shown in fig. 1, the method comprises the steps of:
The invention mainly relates to an intelligent driving function performance test case, a simulation test platform and an evaluation system. The intelligent driving controller and the simulation test platform can be automatically evaluated from the aspects of function specification, performance index, design operation range and the like. The overall framework flow is shown in fig. 2.
As shown in fig. 2, an automated evaluation method for smart driving may include a simulation system and an evaluation system. The simulation system comprises a simulation scene, an intelligent driving controller and a vehicle model (including a vehicle dynamics model, a vehicle kinematics model and the like). The system requirements comprise a plurality of test cases, and the test cases can comprise case numbers, the function and performance requirements of intelligent driving, the output requirements of test results, a design operation domain (namely key parameters of a simulation scene), the requirements of a simulation system and the like. The function requirements of intelligent driving can be lane change requirements, speed reduction requirements and the like. The performance requirements of intelligent driving can be curves which need to be met by lane changing, whether the speed reduction speed meets the requirements or not, and the like. The output requirements of the test results and the requirements of the simulation system may be expected values of the vehicle state parameters.
The simulation scenario in the simulation system may be determined by the test case column.
The test case is input into the intelligent driving controller, and the intelligent driving controller can determine the control driving parameters of the test case in the simulation scene according to the input function and performance requirements of the intelligent driving of the test case, the output requirements of the test result, the design operation domain and the like.
Controlling the driving parameters corresponds to controlling the motion of the vehicle, which may be left-hand steering, stepping on the accelerator, etc. The corresponding control driving parameters may be angle of left turn, acceleration.
The vehicle model is equivalent to an intelligent driving vehicle to be tested, the control driving parameters are input into the vehicle model to be equivalent to the driving test of the intelligent driving vehicle to be tested, and the vehicle state parameters output by the vehicle model are the state of the vehicle after the driving is finished. As an example, the vehicle state parameter may be the current speed of the vehicle, the location in the simulated scene, etc.
And 102, determining whether the simulation system is normal or not according to the test case, the control driving parameters and the vehicle state parameters.
As shown in fig. 2, as an example, after the control driving parameters are input to the vehicle model, the vehicle state parameters are obtained from the vehicle model output. Expected vehicle state parameters may be included in the test case. Therefore, the matching of the vehicle state parameters can be obtained according to the expected vehicle state parameters and the output of the vehicle model, and if the matching is unsuccessful, the simulation system is determined to be abnormal. The test case may also include expected control driving parameters, and if the expected control driving parameters are not matched successfully with the control driving parameters of the input vehicle model, it is determined that the simulation system is abnormal.
And 103, if the test case is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
If the simulation system is normal, the test of the test case needs to be waited for. As an example, the testing time of each test case may be preset to ensure the integrity and consistency of the testing data, which is beneficial to improve the accuracy of the evaluation of the testing result. The preset evaluation criteria may be defined in the test case. The simulation system and the evaluation system run simultaneously, and linkage and real-time performance of simulation test and simulation evaluation are realized.
As shown in fig. 2, the evaluation system includes a signal processor, an observer, an evaluator, and the like. The evaluator evaluates cases except the test case which causes the test failure of the simulation system, and outputs a test result.
According to the automatic evaluation method for intelligent driving disclosed by some embodiments of the invention, the control driving parameters of the test case in the simulation scene are determined by inputting the test case into the intelligent driving controller, and the control driving parameters are input into the vehicle model to obtain the vehicle state parameters, so that the test period of the simulation test work of a huge test scene library for intelligent driving is shortened, and the test efficiency is improved. Whether the simulation system is normal or not is determined according to the test case, the control driving parameters and the vehicle state parameters, if so, the test case is waited to finish the test to obtain a test result, and the test result is evaluated according to a preset evaluation standard, so that the simulation test and the evaluation can be effectively coordinated and parallel, the overall linkage and the real-time performance of the simulation test and the evaluation are improved, the simulation system and the test result are respectively evaluated, and the preliminary classification and the positioning of problems possibly occurring in the driving test process are realized.
In some optional implementations, obtaining a test case, inputting the test case into the intelligent driving controller, and determining control driving parameters of the test case in a simulation scenario includes: the method comprises the steps of obtaining a test case, determining parameters of a simulation scene according to the test case, inputting the parameters of the simulation scene and the test case into an intelligent driving controller, and determining control driving parameters of the test case in the simulation scene.
As an example, after the test case is obtained, the pre-stored simulation scene may be obtained according to the number matching of the test case, the parameters of the simulation scene (or called the design operation domain of the simulation scene) are obtained, and the parameters of the simulation scene and the test case are input to the intelligent driving controller.
In some optional implementation manners, obtaining a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters, including: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case at the current moment under a simulation scene, and inputting the control driving parameters at the current moment into a vehicle model to obtain vehicle state parameters at the current moment; inputting the vehicle state parameters at the current moment into an intelligent driving controller, determining the control driving parameters corresponding to the vehicle state parameters at the current moment at the next moment under a simulation scene, and inputting the control driving parameters at the next moment into a vehicle model to obtain the vehicle state parameters at the next moment; and circulating the steps until the test case finishes the test.
A test case may take a period of time to complete testing in the corresponding simulation scenario. For example, a simulation scenario is a left-turn intersection, i.e., the vehicle needs to go straight for a distance and then turn left. When the parameters of a test case and a simulation scenario are input into the intelligent driving controller, the intelligent driving controller needs to determine an operation action (i.e. control driving parameters, such as straight movement and deceleration) according to the parameters, such as the position and the speed of the initial vehicle, defined in the test case and the current simulation scenario (such as straight movement) faced by the vehicle, and input the control driving parameters into the vehicle model, which is equivalent to driving the vehicle to decelerate straight movement according to the operation action. At the same time, the vehicle state changes (the changed vehicle state is the current vehicle state parameter, and the changed vehicle state may be a straight-ahead, speed-decreasing vehicle state), and the changed vehicle state is the current vehicle state parameter. During the running process of the vehicle in the simulation scene, the specific position of the vehicle in the simulation scene also changes (for example, after the vehicle runs straight, the vehicle is currently in a state of waiting for left turn), so the intelligent driving controller needs to determine the control driving parameter (for example, left turn) at the next moment according to the vehicle state parameter at the current moment and the simulation scene (for example, waiting for left turn) faced by the current vehicle, and the process is repeated until the simulated driving in the simulation scene is completed.
As an example, the time of the simulated driving, within which the simulated driving is completed, may be preset in each test case.
In some optional implementations, determining whether the simulation system is normal according to the test case, the control driving parameter, and the vehicle state parameter includes: determining whether the simulation system is normal or not according to the test case, the control driving parameters at the current moment and the vehicle state parameters at the current moment; and/or determining whether the simulation system is normal according to the test case, the control driving parameters at the next moment and the vehicle state parameters at the next moment.
And judging whether the simulation system is normal or not according to the control driving parameters and the vehicle state parameters obtained in each cycle.
In some optional implementations, determining whether the simulation system is normal according to the test case, the control driving parameter, and the vehicle state parameter includes: and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal.
The observer is used for determining whether the simulation system is normal. As shown in fig. 2, the observer may determine whether the simulation system is normal according to the control driving parameters and the vehicle state parameters output by the intelligent driving controller and the vehicle model, and the test case. As shown in fig. 2, a signal for controlling driving parameters sent by the intelligent driving controller and a signal for controlling vehicle state parameters sent by the vehicle model are input into the signal processor, the signal processor analyzes and sends the signals to the observer according to a preset rule, meanwhile, the observer receives a test case, and determines whether the simulation system is normal or not according to the received test case, the control driving parameters and the vehicle state parameters. As an example, the control driving parameters are straight-going and deceleration, the vehicle state parameters are straight-going and deceleration, and the expected state of the vehicle included in the test case is straight-going and deceleration, so the observer can judge that the vehicle state parameters execute the control driving parameters and the execution is valid, which indicates that the vehicle model is normal; the expected vehicle state included in the test case is consistent with the vehicle state parameters, which shows that the instruction sent by the intelligent driving controller is correct and the intelligent driving controller is normal. Thereby simulating the system to be normal.
In some optional implementations, the observer includes a system state observer, a scene observer, and a communication diagnostor; and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal, wherein the method comprises the following steps: inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the state of the simulation system is normal or not; inputting parameters of the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal; and inputting signals of a preset communication protocol and vehicle state parameters into a communication diagnostor to determine whether the communication is normal or not.
In some embodiments, the intelligent driving performance test cases are written from the system requirement document (the system requirement document includes a plurality of test cases), and each test case includes a case number, a description (including key evaluation index information), an expected result (including key evaluation index information), and the like. The simulation system determines a simulation scene according to a specific case, and parameters of the simulation scene (the parameters of the simulation scene change along with the change of the vehicle state, for example, after the vehicle travels forward for a certain distance, the vehicle may enter an intersection of the simulation scene at the moment, so that the parameters of the simulation scene add traffic light information) are injected into an algorithm in the intelligent driving controller, and further act on a vehicle model. The vehicle model and the intelligent driving controller both output real-time signals to a signal processor of the evaluation system. The signal value processed by the signal processor is output to the observer, the observer can judge whether the test fails due to the fact that the simulation system has problems in the current test case, and the observer can cut in from three aspects of system state, scene and communication diagnosis.
As shown in fig. 2, the evaluation system includes a signal processor, an observer, an evaluator, and the like. The signal processor can analyze and process the output signal of the vehicle and the signal output by the intelligent driving controller. The observer (or called simulation system observer) can determine whether the simulation system is normal or not, comprises a simulation system state observer, a scene observer and a communication diagnostor, observes the simulation system from the three aspects, and eliminates the problem that the test fails due to the simulation system.
As an example, the control driving parameters are straight running and deceleration, the vehicle state parameters are straight running and deceleration, and the expected vehicle state included in the test case is straight running and deceleration. The control driving parameters are consistent with the vehicle state parameters, which shows that the vehicle model correctly executes the control driving parameters, the control driving parameters are determined according to the simulation scene and the test case, and the simulation scene and the intelligent driving controller are normal under the condition that the control driving parameters are correct. Therefore, the simulation system is in a normal state.
As an example, a simple description about the simulation scenario may be included in the test case, and the description is used to determine whether the acquired simulation scenario is correct, and thus, the simple description about the simulation scenario in the test case and the acquired simulation scenario may be input to a scenario observer to determine whether the scenario observer is normal. In addition, the acquisition of the simulation scene can also be obtained by matching the database according to the simple description of the simulation scene in the test case.
As an example, the preset communication protocol may be a preset range that specifies a transmission cycle, a transmission frequency, and the like of the signal to the vehicle state parameter, and the communication diagnoser may determine whether the signal to the vehicle state parameter conforms to the preset communication protocol.
In some optional implementations, inputting the control driving parameters and the vehicle state parameters into a system state observer, and determining whether the simulation system state is normal, includes: determining a predicted vehicle state parameter at the next moment according to the vehicle state parameter at the current moment and the control driving parameter at the current moment; and acquiring the vehicle state parameter at the next moment, determining a deviation value between the vehicle state parameter at the next moment and the predicted vehicle state parameter at the next moment, and if the deviation value does not meet the preset range, judging that the state of the simulation system is abnormal.
Since the vehicle state parameters and the control driving parameters are obtained in a circulating manner in the simulation system, the vehicle state parameters and the control driving parameters at adjacent moments need to be judged. Because the difference between the vehicle state parameters at the adjacent moments and the control driving parameters is generally not too large, if the difference between the vehicle state parameters at the adjacent moments is large, the state of the simulation system is abnormal.
In some embodiments, the observer may obtain the vehicle state information from the vehicle model, and estimate the vehicle state value at the current time (time k + 1) from the vehicle state at the previous time (time k), as follows. And comparing the vehicle state value at the current moment with the actual vehicle measured value, and if the deviation exceeds a reasonable range, determining that the simulation system has a problem, wherein the test case result is failure caused by the simulation system. Otherwise, the simulation system has no problem, and the evaluator evaluates the test case after the test of the current test case is finished.
Wherein, the first and the second end of the pipe are connected with each other,
v: (velocity) vehicle speed
a: (acceleration) acceleration
j: (jerk) acceleration rate of change
Phi: (yaw rate) yaw rate
F k : state transition matrix
u k : external input matrix (Intelligent steering controller command input matrix, e.g. acceleration, steering wheel angle)
As an example, the intelligent driving controller outputs control signals such as a steering lamp, a brake lamp, a wiper and the like to the observer, and meanwhile, if the vehicle model signal received by the observer does not have a corresponding response, the simulation system is considered to have a problem, and the result of the test case is failure caused by the simulation system. Otherwise, the simulation system has no problem, and the evaluator evaluates the test case after the test of the current test case is finished.
In some optional implementation manners, the test case includes a corresponding design operation domain, the simulation scenario also includes a corresponding design operation domain, and the design operation domain is used for representing key parameters of a driving scenario simulated by the test case; and inputting the test case and the simulation scene into a scene observer, and determining whether the simulation scene is normal or not, wherein the steps comprise: and inputting the design operation domain corresponding to the test case and the design operation domain corresponding to the simulation scene into a scene observer, and determining whether the simulation scene is normal.
The design operation domain corresponding to the test case represents key parameters of the simulation scene corresponding to the test case, such as road information of the simulation scene. The simulation scene also comprises a corresponding design operation domain, the design operation domain corresponding to the simulation scene represents key parameters in the simulation scene, such as road information of the simulation scene, and the simulation scene can also comprise other parameters, such as trees and flowers beside the road.
Each test case contains a corresponding design run domain that contains infrastructure (e.g., road types, etc.), driving operational constraints (e.g., speed constraints, etc.), surrounding objects (e.g., signs, etc.), interconnections (e.g., vehicles, etc.), environmental conditions (e.g., weather, lighting, etc.), areas (e.g., geofences, traffic control areas, etc.) that the framework is built under NHTSA. The observer acquires key parameter information (namely a design operation domain corresponding to the test case) in the scene description of the specified test case and corresponding simulation scene information (namely a design operation domain corresponding to the simulation scene) in the simulation system, and compares the design operation domains with the simulation scene information to eliminate test failure caused by scene problems. If the simulation scene is not matched with the scene description of the design operation domain in the test case, the simulation system has problems, the test case result is test failure caused by the simulation system, otherwise, the simulation system has no problems, and the evaluator evaluates after the test of the current test case is finished.
In some optional implementations, the inputting signals of the preset communication protocol and the vehicle state parameter into the communication diagnostor, and the determining whether the communication is normal, comprises: the communication diagnotor determines the transmission period, the signal name and the verification information of the signal according to the received signal of the vehicle state parameter, and checks whether the transmission period, the signal name and the verification information of the signal are normal or not through a preset communication protocol.
The communication diagnotor in the observer can check information such as a signal transmission cycle, a signal name, and check information from the vehicle model according to the communication protocol. If the information is not matched, the simulation system has a communication problem, and the test case result is test failure caused by the simulation system. Otherwise, the simulation system has no problem, and the evaluator evaluates after the test of the current test case is finished.
In some alternative implementations, the evaluation criteria include at least one of: comfort, reliability, fuel economy, safety.
And the evaluator evaluates the current test case according to the evaluation standard, if the test result meets the evaluation standard of the system requirement, the test result is normal, otherwise, the test result is failed, and the tested object intelligent driving controller fails the test. Aiming at a huge intelligent driving scene library, the method can realize the linkage mechanism of an automatic evaluation and simulation system and an automatic evaluation system for the execution result of the test case, can release part of manpower and test resources, improves the test efficiency and shortens the test period.
As shown in fig. 2, when the observer finishes determining whether the simulation system is normal, the primary classification of the failure reasons of the failed test case is realized: one is the problem of no simulation system, namely test failure caused by the problem of the simulation system; one type is a problem with simulation systems, i.e., test failures due to the reason of the intelligent driving controller, so that a test engineer can perform efficient output according to the location of the problem. After eliminating the problem of whether the simulation system is available, the test execution condition of the current case (namely, the control driving parameters and the vehicle state parameters at all the time after the test of the test case is completed) and the test case evaluation standard from the system requirement (the evaluation standard can be recorded in the test case) are injected into the evaluator. The evaluation criteria from the system demand test case include comfort (e.g., acceleration change exceeding a threshold value, indicating poor comfort), reliability (e.g., number of unsuccessful lane changes), fuel economy (e.g., vehicle model may define an initial fuel amount, and fuel condition is calculated based on parameters output by the vehicle model to determine fuel economy), safety (e.g., number of crashes), and other performance indicators. On one hand, the evaluation standard of the tested object (namely the evaluation standard for quantifying whether the algorithm of intelligent driving control is normal) is quantified from the requirement definition level, so that the evaluation of the work performance test result of the intelligent driving controller from different requirements such as comfort, reliability, fuel economy, safety and the like is realized; and on the other hand, the simulation system is subjected to quantitative evaluation of confidence from the perspective of the simulation system.
Referring to fig. 3, fig. 3 is a schematic structural diagram of some embodiments of an automatic evaluation device for intelligent driving according to the present invention, and as an implementation of the methods shown in the above figures, the present invention further provides some embodiments of an automatic evaluation device for intelligent driving, which correspond to some embodiments of the methods shown in fig. 1, and which can be applied to various electronic devices.
As shown in fig. 3, the automated evaluation device for smart driving of some embodiments includes a simulation module 301, a simulation system evaluation module 302, and a smart driving controller evaluation module 303: the simulation module 301 is configured to obtain a test case, input the test case into the intelligent driving controller, determine control driving parameters of the test case in a simulation scene, and input the control driving parameters into a vehicle model to obtain vehicle state parameters; the simulation system evaluation module 302 is used for determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters; and the intelligent driving controller evaluation module 303 is configured to wait for the test case to finish the test if the test case is normal, obtain a test result, and evaluate the test result according to a preset evaluation standard.
In an optional implementation manner of some embodiments, the simulation module 301 is further configured to: the method comprises the steps of obtaining a test case, determining parameters of a simulation scene according to the test case, inputting the parameters of the simulation scene and the test case into an intelligent driving controller, and determining control driving parameters of the test case in the simulation scene.
In an optional implementation manner of some embodiments, the simulation module 301 is further configured to: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case at the current moment under a simulation scene, and inputting the control driving parameters at the current moment into a vehicle model to obtain vehicle state parameters at the current moment; inputting the vehicle state parameters at the current moment into an intelligent driving controller, determining the control driving parameters corresponding to the vehicle state parameters at the current moment at the next moment under a simulation scene, and inputting the control driving parameters at the next moment into a vehicle model to obtain the vehicle state parameters at the next moment; and circulating the steps until the test case finishes the test.
In an alternative implementation of some embodiments, simulation system evaluation module 302 is further configured to: determining whether the simulation system is normal or not according to the test case, the control driving parameters at the current moment and the vehicle state parameters at the current moment; and/or determining whether the simulation system is normal according to the test case, the control driving parameters at the next moment and the vehicle state parameters at the next moment.
In an alternative implementation of some embodiments, simulation system evaluation module 302 is further configured to: and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal.
In an optional implementation of some embodiments, the observer comprises a system state observer, a scene observer, and a communication diagnostor; and, simulation system evaluation module 302, further configured to: inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the state of the simulation system is normal or not; inputting parameters of the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal; and inputting signals of a preset communication protocol and vehicle state parameters into a communication diagnostor to determine whether the communication is normal or not.
In an optional implementation manner of some embodiments, the simulation system evaluation module 302 is further configured to: determining a predicted vehicle state parameter at the next moment according to the vehicle state parameter at the current moment and the control driving parameter at the current moment; and acquiring the vehicle state parameter at the next moment, determining the deviation value between the vehicle state parameter at the next moment and the predicted vehicle state parameter at the next moment, and if the deviation value does not meet the preset range, judging that the state of the simulation system is abnormal.
In an optional implementation manner of some embodiments, the test case includes a corresponding design operation domain, the simulation scenario also includes a corresponding design operation domain, and the design operation domain is used for representing key parameters of a driving scenario simulated by the test case; and, simulation system evaluation module 302, further configured to: and inputting the design operation domain corresponding to the test case and the design operation domain corresponding to the simulation scene into a scene observer, and determining whether the simulation scene is normal.
In an alternative implementation of some embodiments, simulation system evaluation module 302 is further configured to: the communication diagnotor determines the transmission period, the signal name and the verification information of the signal according to the received signal of the vehicle state parameter, and checks whether the transmission period, the signal name and the verification information of the signal are normal or not through a preset communication protocol.
In an alternative implementation of some embodiments, the evaluation criteria includes at least one of: comfort, reliability, fuel economy, safety.
It will be appreciated that the modules described in the apparatus correspond to the steps in the method described with reference to figure 1. Therefore, the operations, features and advantages of the methods described above are also applicable to the apparatus and the modules and units included therein, and are not described herein again.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. Processor 410 may invoke logic instructions in memory 430 to perform an automated assessment method for smart driving, the method comprising: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; determining whether the simulation system is normal or not according to the test case, the control driving parameters and the vehicle state parameters; and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the automated assessment method for smart driving provided by the above methods, the method comprising: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters; and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the automated assessment method for smart driving provided above, the method comprising: acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters; determining whether the simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters; and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. Based on the understanding, the above technical solutions substantially or otherwise contributing to the prior art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (14)
1. An automated assessment method for smart driving, comprising:
acquiring a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters;
determining whether the simulation system is normal or not according to the test case, the control driving parameters and the vehicle state parameters;
and if the test result is normal, waiting for the test case to finish the test to obtain a test result, and evaluating the test result according to a preset evaluation standard.
2. The automated evaluation method for intelligent driving according to claim 1, wherein the obtaining a test case, inputting the test case into an intelligent driving controller, and determining the control driving parameters of the test case in a simulation scenario comprises:
acquiring a test case, determining parameters of a simulation scene according to the test case, inputting the parameters of the simulation scene and the test case into an intelligent driving controller, and determining control driving parameters of the test case in the simulation scene.
3. The automated evaluation method for intelligent driving according to claim 2, wherein the obtaining a test case, inputting the test case into an intelligent driving controller, determining control driving parameters of the test case in a simulation scenario, inputting the control driving parameters into a vehicle model, and obtaining vehicle state parameters comprises:
acquiring a test case, inputting the test case into an intelligent driving controller, determining the control driving parameters of the test case at the current moment under a simulation scene, and inputting the control driving parameters at the current moment into a vehicle model to obtain the vehicle state parameters at the current moment;
inputting the vehicle state parameters at the current moment into an intelligent driving controller, determining the control driving parameters corresponding to the vehicle state parameters at the current moment at the next moment under a simulation scene, and inputting the control driving parameters at the next moment into the vehicle model to obtain the vehicle state parameters at the next moment;
and circulating the steps until the test case finishes the test.
4. The automated evaluation method for smart driving of claim 3, wherein the determining whether a simulation system is normal according to the test case, the control driving parameters, and the vehicle state parameters comprises:
determining whether the simulation system is normal or not according to the test case, the control driving parameters at the current moment and the vehicle state parameters at the current moment; and/or
And determining whether the simulation system is normal or not according to the test case, the control driving parameters at the next moment and the vehicle state parameters at the next moment.
5. The automated evaluation method for intelligent driving according to claim 4, wherein the determining whether a simulation system is normal according to the test case, the control driving parameters and the vehicle state parameters comprises:
and inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether the simulation system is normal.
6. The automated evaluation method for smart driving of claim 5, wherein the observer comprises a system state observer, a scene observer, and a communication diagnostician; and (c) a second step of,
the step of inputting the test case, the control driving parameters and the vehicle state parameters into an observer to determine whether a simulation system is normal comprises the following steps:
inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the state of the simulation system is normal;
inputting the parameters of the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal;
and inputting signals of a preset communication protocol and the vehicle state parameters into a communication diagnostor to determine whether the communication is normal or not.
7. The automated evaluation method for smart driving according to claim 6, wherein the inputting the control driving parameters and the vehicle state parameters into a system state observer to determine whether the simulation system state is normal comprises:
determining a predicted vehicle state parameter at the next moment according to the vehicle state parameter at the current moment and the control driving parameter at the current moment;
and acquiring the vehicle state parameter at the next moment, determining the deviation value between the vehicle state parameter at the next moment and the predicted vehicle state parameter at the next moment, and if the deviation value does not meet the preset range, judging that the state of the simulation system is abnormal.
8. The automated evaluation method for intelligent driving according to claim 6, wherein the test case comprises a corresponding design operation domain, and the simulation scenario also comprises a corresponding design operation domain, and the design operation domain is used for representing key parameters of the driving scenario simulated by the test case; and (c) a second step of,
inputting the test case and the simulation scene into a scene observer to determine whether the simulation scene is normal, wherein the method comprises the following steps:
and inputting the design operation domain corresponding to the test case and the design operation domain corresponding to the simulation scene into a scene observer to determine whether the simulation scene is normal.
9. The automated assessment method for intelligent driving according to claim 6, wherein the inputting of signals of preset communication protocols and the vehicle state parameters into a communication diagnostor to determine whether the communication is normal comprises:
and the communication diagnotor determines the transmission period, the signal name and the verification information of the signal according to the received signal of the vehicle state parameter, and checks whether the transmission period, the signal name and the verification information of the signal are normal or not through a preset communication protocol.
10. The automated evaluation method for smart driving of claim 1, wherein the evaluation criteria comprises at least one of: comfort, reliability, fuel economy, safety.
11. An automated evaluation device for smart driving, comprising:
the simulation module is used for acquiring a test case, inputting the test case into the intelligent driving controller, determining control driving parameters of the test case in a simulation scene, and inputting the control driving parameters into a vehicle model to obtain vehicle state parameters;
the simulation system evaluation module is used for determining whether the simulation system is normal or not according to the test case, the control driving parameters and the vehicle state parameters;
and the intelligent driving controller evaluation module is used for waiting for the test case to finish the test if the test case is normal, obtaining a test result and evaluating the test result according to a preset evaluation standard.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, carries out the steps of the automated assessment method for intelligent driving according to any one of claims 1 to 10.
13. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the automated assessment method for intelligent driving according to any one of claims 1 to 10.
14. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the automated assessment method for intelligent driving according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210699670.1A CN115129027A (en) | 2022-06-20 | 2022-06-20 | Automatic evaluation method and device for intelligent driving |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210699670.1A CN115129027A (en) | 2022-06-20 | 2022-06-20 | Automatic evaluation method and device for intelligent driving |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115129027A true CN115129027A (en) | 2022-09-30 |
Family
ID=83380031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210699670.1A Pending CN115129027A (en) | 2022-06-20 | 2022-06-20 | Automatic evaluation method and device for intelligent driving |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115129027A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117369407A (en) * | 2023-10-30 | 2024-01-09 | 载合汽车科技(苏州)有限公司 | Automobile electrical performance test system, method and device and storage medium |
-
2022
- 2022-06-20 CN CN202210699670.1A patent/CN115129027A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117369407A (en) * | 2023-10-30 | 2024-01-09 | 载合汽车科技(苏州)有限公司 | Automobile electrical performance test system, method and device and storage medium |
| CN117369407B (en) * | 2023-10-30 | 2024-05-10 | 载合汽车科技(苏州)有限公司 | Automobile electrical performance test system, method and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795818B (en) | Method and device for determining virtual test scene, electronic equipment and storage medium | |
| CN109991987B (en) | Automatic driving decision-making method and device | |
| US20170132117A1 (en) | Method and device for generating test cases for autonomous vehicles | |
| EP3958129A1 (en) | Method and system for validating autonomous control software for a self-driving vehicle | |
| CN111273644B (en) | Automatic parking active braking test method based on CAN bus programming | |
| WO2023116344A1 (en) | Driverless driving test method, driverless driving test system, and computer device | |
| CN112613169A (en) | Expected function safety analysis method for misoperation of automatic driving vehicle | |
| CN110803172B (en) | Modeling method of brake system of automatic driving vehicle and vehicle brake system | |
| US20220198107A1 (en) | Simulations for evaluating driving behaviors of autonomous vehicles | |
| CN112668012B (en) | Method and system for verifying safety of automatic driving human-computer interface | |
| CN113536611B (en) | Method for checking correctness of autonomous traffic system architecture based on discrete simulation | |
| CN112987711B (en) | Optimization method of automatic driving regulation algorithm and simulation testing device | |
| CN109094574B (en) | Unmanned vehicle driving state measurement control system based on rack | |
| CN115599694A (en) | Integrated test method for automatic driving software version | |
| CN111765904B (en) | Test method and device for automatic driving vehicle, electronic equipment and medium | |
| CN115129027A (en) | Automatic evaluation method and device for intelligent driving | |
| CN115113542A (en) | Automatic driving simulation method, system, electronic device and readable storage medium | |
| CN111983934B (en) | Unmanned vehicle simulation test case generation method and system | |
| CN114647584A (en) | Control method and system for automobile micro environment and readable storage medium | |
| KR20200082672A (en) | Simulation method for autonomous vehicle linked game severs | |
| US20220358024A1 (en) | Computer-implemented method for scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested by means of key performance indicators (kpi) | |
| CN115358415A (en) | Distributed training method of automatic driving learning model and automatic driving method | |
| KR102553075B1 (en) | Apparatus and method for simulating interactive ldm-hil | |
| US20240169122A1 (en) | Systems and methods for optimized vehicular simulations | |
| DE102022127006A1 (en) | RATING SYSTEM FOR AUTONOMOUS DRIVING |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |