CN112462759B - Evaluation method, system and computer storage medium of rule control algorithm - Google Patents
Evaluation method, system and computer storage medium of rule control algorithm Download PDFInfo
- Publication number
- CN112462759B CN112462759B CN202011262736.8A CN202011262736A CN112462759B CN 112462759 B CN112462759 B CN 112462759B CN 202011262736 A CN202011262736 A CN 202011262736A CN 112462759 B CN112462759 B CN 112462759B
- Authority
- CN
- China
- Prior art keywords
- scene
- data
- abnormal
- evaluation
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 139
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 230000002159 abnormal effect Effects 0.000 claims abstract description 155
- 238000000034 method Methods 0.000 claims description 26
- 238000004891 communication Methods 0.000 claims description 17
- 238000013136 deep learning model Methods 0.000 claims description 7
- 238000004140 cleaning Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 230000007613 environmental effect Effects 0.000 claims description 5
- 230000001105 regulatory effect Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 2
- 230000006399 behavior Effects 0.000 description 16
- 238000004088 simulation Methods 0.000 description 14
- 238000012795 verification Methods 0.000 description 10
- 230000001133 acceleration Effects 0.000 description 8
- 230000004913 activation Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000004927 fusion Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 208000012661 Dyskinesia Diseases 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 230000008929 regeneration Effects 0.000 description 1
- 238000011069 regeneration method Methods 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
Abstract
The invention provides an evaluation method and system of a regulation algorithm and a computer storage medium, wherein the evaluation method of the regulation algorithm comprises the following steps: the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data; after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line; and the offline end constructs a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene based on the virtual scene. The performance of the regulation algorithm under abnormal conditions is evaluated, and the performance of the regulation algorithm is improved.
Description
Technical Field
The present invention relates to the field of automatic driving, and in particular, to a method and a system for evaluating a rule control algorithm, and a computer storage medium.
Background
The advantages and disadvantages of the regulation algorithm directly affect the performance of the intelligent driving automobile. At present, two modes are generally adopted when performance evaluation is carried out on the rule control algorithm, and real vehicle verification or digital means simulation is adopted. The actual effect of the algorithm can be more intuitively represented by the actual vehicle verification, but the actual vehicle verification has the defects of small scene repeatability, large resource consumption and the like. The digital simulation method is the best choice at present, the feasibility of the algorithm can be rapidly verified when the algorithm is modified in a small scale, the problem of more than 90% is avoided in the early stage of algorithm development, and in addition, the digital simulation method has excellent scene reproduction capability and high consistency. However, the existing evaluation method of the regulation algorithm lacks comprehensive evaluation of various driving scenes of the regulation algorithm, particularly lacks performance evaluation under abnormal conditions, and the performance of the algorithm under the abnormal conditions is just the tripping stone of the current high-level automatic driving.
Disclosure of Invention
In view of this, the invention provides an evaluation method, an evaluation system and a computer storage medium for a regulation algorithm, which can evaluate the performance of the regulation algorithm under abnormal conditions, and is beneficial to improving the performance of the regulation algorithm.
In a first aspect, the present invention provides a method for evaluating a rule control algorithm, including:
the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data;
after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line;
and the offline end constructs a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene based on the virtual scene.
The vehicle end identifies an abnormal scene according to vehicle data, and the method comprises the following steps:
judging whether the vehicle data accords with preset conditions or not;
if yes, confirming that the scene corresponding to the vehicle data is an abnormal scene;
performing preliminary evaluation on the rule control algorithm of the abnormal scene to obtain a first evaluation result;
and determining whether to transmit the abnormal scene data to the offline end according to the first evaluation result.
After the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the line lower end, wherein the method comprises the following steps of:
acquiring a data acquisition reference time point;
acquiring external data of 2n+5 minutes in the front n minutes and the rear n+5 minutes of the data acquisition reference time point;
generating abnormal scene data according to the external data, the first evaluation result and a high-precision map;
and optimally compressing the abnormal scene data, and transmitting the abnormal scene data to the offline end through road side communication.
The offline end builds a virtual scene according to the abnormal scene data, and the virtual scene comprises:
cleaning the external data to obtain physical characteristics of dynamic obstacles in the abnormal scene;
inputting the cleaned external data and the high-precision map into a deep learning model to obtain a plurality of groups of abnormal scenes including dynamic obstacles under different environmental information;
and constructing a virtual scene according to the multiple groups of abnormal scenes.
After obtaining the abnormal scene including the dynamic obstacle under the plurality of sets of different environmental information, the method further comprises:
screening, quantifying and grading the multiple groups of abnormal scenes;
acquiring the preset passing rate corresponding to the levels of the screened multiple groups of abnormal scenes;
and comparing the relation between the actual passing rate of the rule algorithm and the preset passing rate to obtain a second evaluation result.
Wherein after the virtual scene is constructed according to the multiple groups of abnormal scenes, the method further comprises:
combining the multiple groups of abnormal scenes and vehicle data of the abnormal scenes to generate virtual abnormal scenes;
registering the virtual abnormal scene to a scene library.
Wherein, the evaluating the rule algorithm corresponding to the abnormal scene in the virtual scene further includes:
evaluating a rule control algorithm in the virtual scene by adopting at least two evaluation methods of rule evaluation, model evaluation and manual evaluation;
and carrying out weighted average on the evaluation results of at least two evaluation methods of rule evaluation, model evaluation and manual evaluation to obtain a third evaluation result of the rule control algorithm.
In a second aspect, the invention also provides an evaluation system of the rule control algorithm, which comprises a vehicle end, an off-line end and a road side communication module;
the vehicle end is used for identifying an abnormal scene according to vehicle data and generating abnormal scene data, wherein the vehicle data comprises external data and internal data;
the road side communication module is used for transmitting the abnormal scene data to the lower end of the line;
the offline end is used for constructing a virtual scene according to the abnormal scene data and evaluating a rule control algorithm corresponding to the abnormal scene based on the virtual scene.
Wherein, the evaluation system of the rule algorithm further comprises:
and the high-precision map module is used for storing high-precision map information of the area required by the automatic driving area and providing map information for the automatic driving and evaluating module.
In a third aspect, the present invention also provides a computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement the method of evaluating a regulatory algorithm as described above.
In summary, the method, the system and the computer storage medium for evaluating the control algorithm of the present invention comprise: the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data; after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line; and the offline end constructs a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene based on the virtual scene. The performance of the regulation algorithm under abnormal conditions is evaluated, and the performance of the regulation algorithm is improved.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention, as well as the preferred embodiments thereof, together with the following detailed description of the invention, given by way of illustration only, together with the accompanying drawings.
Drawings
FIG. 1 is a flow chart of an evaluation method of a rule control algorithm according to a first embodiment;
FIG. 2 is a block diagram of an evaluation system of the regulation algorithm according to the second embodiment;
fig. 3 is a structural view of a wire lower end shown according to a second embodiment;
fig. 4 is a schematic view of a virtual scene shown according to a third embodiment.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description of the present invention is given with reference to the accompanying drawings and preferred embodiments.
First embodiment
Fig. 1 is a flowchart of an evaluation method of the rule and control algorithm according to the first embodiment. As shown in fig. 1, an embodiment of the present invention provides a method for evaluating a rule control algorithm, including:
step 201: the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data;
step 202: after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line;
step 203: and the offline end constructs a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene based on the virtual scene.
The actual traffic situation is complex, and the information related to an abnormal scene is less in the general data acquisition process, so the invention evaluates the performance of the regulation algorithm in the abnormal scene (a marker case, an unexpected event), such as sudden crossing, sudden deceleration of a front vehicle and the like, and can evaluate multiple angles in different modules to obtain evaluation results, thereby providing the evaluation accuracy. In step 201, the vehicle end identifies an abnormal scene according to the vehicle data. The vehicle data comprises external data and internal data, the external data comprises data of each hardware sensor of the vehicle end received by the sensing module, and the internal data comprises vehicle chassis information and the like. Then, the vehicle end judges whether the acquired vehicle data accords with preset conditions for judging the abnormal scene through an abnormal scene recognition module; if yes, identifying the scene corresponding to the vehicle data as an abnormal scene and activating the vehicle end evaluation module. Whether the current scene is an abnormal scene or not can be judged according to parameters such as a running track, a vehicle speed, a road gradient, a steering wheel rotation amplitude, a frequency and the like obtained by vehicle data. If the curvature smoothness of the running track of the vehicle, the smoothness of the speed and acceleration curves and the road gradient are larger than or equal to a preset threshold value, judging that the current scene is an abnormal scene. For example, in the normal driving process, the vehicle suddenly traverses in front, at this time, the vehicle needs to slow down or adjust the steering wheel angle immediately, and after corresponding vehicle data is collected, the current scene can be judged to be an abnormal scene according to the speed change of the internal data in the vehicle data, the steering wheel change and the movement curve of the front obstacle in the external data.
In step 202, after the vehicle end identifies the abnormal scene, when generating the abnormal scene data and transmitting the abnormal scene data to the line lower end, a first evaluation result of the rule control algorithm by the vehicle end evaluation module needs to be obtained. The vehicle end performs preliminary evaluation on the regulation algorithm of the abnormal scene, the difference between the vehicle data and the preset threshold is compared and evaluated to obtain a first evaluation result, whether the abnormal scene is a high-value scene is judged according to the first evaluation result, and the larger the difference between the vehicle data and the preset threshold is, the higher the value of the abnormal scene is. If the abnormal scene is a high-value scene, determining to transmit the abnormal scene data to the offline end. And acquiring external data of 2n+5 minutes which is n minutes before and n+5 minutes after the data acquisition reference time point by using the time point of activation of the vehicle end evaluation module as the data acquisition reference time point, generating abnormal scene data by finally generating an evaluation result, the external data, a high-precision map number and a position ID, optimally compressing the abnormal scene data, and transmitting the abnormal scene data to the offline end through roadside communication. Specifically, the storage module with fixed duration stores all sensing modules outputting data in the total of 2n+5 minutes of the first n minutes and the last n+5 minutes when the evaluation module is activated, and the module adopts a double backup means by taking minutes as a unit to ensure the data storage stability. The storage time is determined by the storage hardware device used. And then, the vehicle end evaluation module receives the evaluation activation module information, obtains scene information from the fixed duration, and performs online evaluation on the performance of the regulation algorithm. And the high-precision map information of the area required by the automatic driving area is stored by the high-precision map module, so that detailed map information is provided for the automatic driving and evaluating module. And finally, the data transmission module compresses and stores the perceived data and the evaluation result into abnormal scene data, the abnormal scene data are transmitted to the road side communication modules deployed in the abnormal scene generation area of probability estimation and each traffic main flow crossing point, and the road side communication modules transmit the abnormal scene data to the offline end.
In step 203, when the offline end constructs a virtual scene according to the abnormal scene data, the sensor data needs to be cleaned to obtain the physical characteristics of the dynamic barrier in the abnormal scene; then inputting the cleaned external data and the high-precision map into a deep learning model to obtain a plurality of groups of abnormal scenes including dynamic obstacles under different environmental information; and constructing a virtual scene according to the multiple groups of abnormal scenes. After constructing a virtual scene according to a plurality of groups of abnormal scenes, combining the plurality of groups of abnormal scenes with vehicle data of the abnormal scenes to generate a virtual abnormal scene; registering the virtual abnormal scene to the scene library. Specifically, the off-line end cleans data in the real vehicle collected data through a data cleaning module to obtain physical characteristics (length, width, height, human body characteristic points and the like) and target behaviors (acceleration, deceleration, turning, track and the like) of perceived dynamic obstacles. The model expansion module inputs the cleaned data as initial data and a high-precision map into the deep learning model, extracts environment information in the initial data, expands initial dynamic target behaviors into other environments, generalizes target actions and generates various dynamic target behavior scenes under a plurality of environments. The simulation evaluation module evaluates the behaviors of each extended scene, eliminates scenes with similar behaviors, classifies abnormal scenes in the scenes according to the passing difficulty of vehicles, can be divided into 5 stages, wherein different stages correspond to different preset passing rates, the 1-2 stages are necessary passing scenes, the 3-stage passing rate needs to reach 95%, the 4-stage scene passing rate needs to reach 90%, the 5-stage scene passing rate needs to reach 85%, and the abnormal scenes are used as reference evidence of a subsequent digital simulation result, namely a second evaluation result. The vehicle database module then stores the vehicle profile as well as the vehicle initial state (speed, acceleration, heading, etc.). The scene library registration module receives the simulation evaluation module and the vehicle database module, constructs a virtual scene and registers the virtual scene into a scene library as an example of algorithm evaluation. The verification algorithm input module is used for accessing a rule control algorithm to be verified, and realizing data type unification and conversion.
When a second evaluation result is obtained, firstly, carrying out quantization classification on a plurality of groups of abnormal scenes; acquiring preset passing rates corresponding to the levels of a plurality of groups of abnormal scenes; and evaluating the relation between the actual passing rate and the preset passing rate of the rule algorithm. The scene grade is output through the simulation evaluation module, the 1-2 grade is the scene which must pass, the 3 grade passing rate is required to reach 95%, the 4 grade scene passing rate is required to reach 90%, the 5 grade scene passing rate is required to reach 85%, and the passing rate refers to the proportion of the scene which is tested to pass by the rule control algorithm to all the scenes to be tested in the same grade. If the actual passing rate of the estimated regulation algorithm is greater than or equal to the passing rate of the corresponding scene grade, judging that the regulation algorithm is qualified, otherwise, disqualification. For example, the simulation evaluation module generalizes 100 abnormal scenes, 40 of which are of 1-2 levels, and the passing rate is 100%, namely, the rule algorithm is qualified when all tests pass in the 40 scenes; the 20 are 3 stages, and the pass rate of the rule control algorithm is more than or equal to 95 percent and is qualified; the 20 are 4-level, and the pass rate of the rule control algorithm is more than or equal to 90 percent and is qualified; the 20 standard control algorithms are 5-level, the passing rate of the standard control algorithms is more than or equal to 85%, the second evaluation result can be used as a verification evaluation means for the corrected standard control algorithms, and the passing rate of the corrected standard control algorithms in different scenes is guaranteed to reach the standard.
In step 203, the rule control algorithm corresponding to the abnormal scene is evaluated in the virtual scene, and the method further includes: and carrying out weighted average on the evaluation results of at least two evaluation methods in the rule evaluation, the model evaluation and the manual evaluation to obtain an evaluation value of the rule control algorithm as a third evaluation result. And when the weighted evaluation value of the rule algorithm is larger than a preset value, judging that the rule algorithm is qualified, otherwise, judging that the rule algorithm is unqualified, and further training and optimizing are needed, wherein the corrected rule algorithm can obtain a second evaluation result through the simulation evaluation module to evaluate the pass rate of the second evaluation result, so that the verification of whether the corrected rule algorithm is qualified is facilitated. For example, when the weighted evaluation value of the regulation algorithm is 90 minutes and the preset value is 85 minutes, the regulation algorithm is qualified. The algorithm evaluation module is connected with the verification algorithm access module and traverses the scene library or the classification scene designated by the user. Three main functions are built in the evaluation module: based on rule evaluation, based on model evaluation and based on manual evaluation. Based on rule evaluation, including system classes: whether the algorithm has breakdown, algorithm time delay and the like; an Quanlei: whether the vehicle collides, etc.; and (3) traffic rule class: judging whether the current intelligent driving vehicle has the behavior of violating the traffic rules, such as line pressing, overspeed and the like based on the high-precision map; somatosensory class: whether the emergency brake exists, whether the centrifugal sense is strong, and the like; the algorithm class: the stability of the control algorithm output, the degree of phase difference from the expected output of the planning algorithm, and the like. Based on the model evaluation, comprising: based on a learning model, the rule-based output is learned, and the result is corrected by assisting with manual evaluation. Based on the manual evaluation, comprising: and the user inputs subjective scores of the regular algorithm expression on the result visual interface, and inputs training parameters serving as a model, so that the evaluation performance of the model is further improved.
According to the embodiment of the invention, the communication module is deployed for the abnormal scene which is easy to occur by combining with data analysis, the abnormal scene identification module is deployed at the vehicle end, the vehicle end completes the evaluation of the rule control algorithm, and the scene data with higher value is synchronized to the offline end through the communication module, so that the performance of the on-line research algorithm is further improved. Meanwhile, based on a high-precision map, the abnormal scene is restored, real dynamic target behavior data are collected, a deep learning model is used for expanding feasible abnormal behaviors, and the abnormal scene is enriched. In addition, the algorithm evaluation result is obtained by using a rule evaluation module, a model evaluation module and a subjective evaluation module, and the abnormal normal control algorithm is comprehensively evaluated by a fusion evaluation method.
The evaluation method of the regulation algorithm provided by the embodiment of the invention comprises the following steps: the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data; after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line; and the offline end constructs a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene in the virtual scene. The performance of the regulation algorithm under abnormal conditions is evaluated, and the performance of the regulation algorithm is improved.
Second embodiment
Fig. 2 is a block diagram of an evaluation system of the regulation algorithm according to the second embodiment. Fig. 3 is a structural view of a line lower end shown according to a second embodiment. As shown in fig. 2 and 3, the evaluation system of the rule control algorithm comprises a vehicle end, an off-line end and a road side communication module; the vehicle end is used for identifying an abnormal scene according to vehicle data and generating abnormal scene data, wherein the vehicle data comprises external data and internal data; the road side communication module is used for transmitting the abnormal scene data to the lower end of the line; and the offline end is used for constructing a virtual scene according to the abnormal scene data and evaluating a rule control algorithm corresponding to the abnormal scene in the virtual scene.
Specifically, the vehicle end comprises a sensing module, an evaluation activation module, a fixed duration storage module, a vehicle end evaluation module and a high-precision map module. The sensing module receives the data of each hardware sensor at the vehicle end and performs fusion, cleaning and optimization processing on each data. The evaluation activation module receives the data output by the sensing module and the vehicle chassis information, judges the current data through the built-in model, defines whether the scene is an abnormal scene or not, and activates the vehicle end evaluation module. The fixed duration storage module stores all sensing modules which output data in total of 2n+5 minutes from the first n minutes and the last n+5 minutes when the evaluation module is activated, and the module takes minutes as a unit, adopts a double backup means, and ensures stable data storage. The storage time is determined by the storage hardware device. And the vehicle end evaluation module receives the information of the evaluation activation module, obtains scene information from the fixed duration and performs online evaluation on the performance of the regulation algorithm. The high-precision map module stores high-precision map information of an area required by the automatic driving area and provides detailed map information for the automatic driving and evaluating module. The data transmitting module compresses and stores the perceived data and the evaluation result and transmits the compressed and stored perceived data and the evaluation result to the road side communication module.
The road side communication module is deployed in the abnormal scene occurrence area of probability estimation and each traffic main flow crossing point.
The off-line end comprises a data cleaning module, a model expansion module, a simulation evaluation module, a vehicle database module, a scene library registration module, a verification algorithm input module, an algorithm evaluation module and a result visualization module. The data cleaning module cleans data in the data acquired by the real vehicle to obtain physical characteristics (length, width, height, human body characteristic points and the like) and target behaviors (acceleration, deceleration, turning, track and the like) of the perceived dynamic obstacle. The model expansion module inputs the cleaned data as initial data and a high-precision map into the deep learning model, extracts environment information in the initial data, expands initial dynamic target behaviors into other environments, generalizes target actions and generates various dynamic target behavior scenes under a plurality of environments. The simulation evaluation module evaluates the behaviors of each extended scene, eliminates scenes with similar behaviors, and classifies abnormal states of the scenes to be used as reference credentials of subsequent digital simulation results. The vehicle database module stores vehicle profiles and vehicle initial states (speed, acceleration, heading, etc.). The scene library registration module receives the simulation evaluation module and the vehicle database module, constructs a virtual scene and registers the virtual scene into a scene library as an example of algorithm evaluation. The verification algorithm input module is used for accessing a rule control algorithm to be verified, and realizing data type unification and conversion. The algorithm evaluation module is connected with the verification algorithm access module and traverses the scene library or the classification scene designated by the user. Three main functions are built in the evaluation module: based on rule evaluation, based on model evaluation and based on manual evaluation. And the result visualization module is used for presenting the offline process and the result to a user in an interface form.
According to the embodiment of the invention, the performance of the regulation algorithm in an abnormal scene is evaluated in real time through the vehicle end. And by combining a communication module, scene resources are shared, and the performance of the research regulation algorithm is promoted. Based on the high-precision map, the real dynamic target behavior is sampled, and the scene built offline is closer to the real world. And a deep learning model is adopted to expand an abnormal scene, and the performance capability of the regulation algorithm under an extreme scene is tested. And the multi-dimensional method is adopted for fusion evaluation, so that the evaluation of the digital simulation result is more reliable.
The specific process of executing the above steps in this embodiment is described in detail in the above embodiments, and will not be repeated here.
Third embodiment
Fig. 4 is a schematic view of a virtual scene shown according to a third embodiment. As shown in fig. 4, the original scenario may be that in a three lane environment, a left front vehicle located in a first lane suddenly crosses a lane to a third lane, interfering with an intelligent driving vehicle located in a second lane. The vehicle end evaluation module receives the abnormal movement of the vehicle chassis data and combines the perceived data to activate the vehicle end evaluation module to evaluate the performance of the current regulation algorithm. And the evaluation result, the perception data, the high-precision map number and the position ID are optimally compressed and transmitted to the lower end of the loop through the road side communication.
And processing data at the lower end of the line to obtain the physical characteristics (length, width and height) and the behavior characteristics (speed, orientation, speed track and driving track) of the obstacle vehicle. The method comprises the steps of analyzing the current environment (lane information, virtual white line information, crossing distance and the like), copying the current scene into other map scenes (such as a bidirectional lane, a crossing, a left lane line and a right lane line which are different and the like), generalizing dynamic targets (target end points, simple uniform acceleration and deceleration, regeneration paths based on speed and the like) according to the acquired physical characteristics, eliminating behaviors which do not affect intelligent driving such as driving in a non-high-precision map range and behaviors with maximum acceleration and deceleration exceeding 7.6m/s and the like exceeding actual bearing capacity through an analog evaluation module, and quantitatively evaluating the generalized scenes (1-5 points, 1 grade is lowest and 5 grade is highest). And generating an abnormal scene (incorrect scenes such as elimination coincidence, too close distance from the obstacle, no reaction time and the like) according to the vehicle configuration and different initial states (the three-dimensional size, the initial speed, the initial acceleration and the initial direction of the initial vehicle), and registering the abnormal scene in a scene library. The generated scenes can be a three-lane-left-side vehicle crossing to right-lane-accelerating scene, a three-lane-left-side vehicle crossing to right-lane-decelerating-self-vehicle initial speed zero scene, a three-lane-left-side vehicle crossing to right-lane-accelerating-self-vehicle speed 10m/s scene, an intersection-left-side vehicle crossing to right-lane-accelerating-self-vehicle-all-accelerating scene, a two-lane-left-side vehicle invading the self-vehicle lane-decelerating-self-vehicle speed 5 m/s-course angle-10 DEG scene, and the like.
And accessing a rule control algorithm required to be evaluated through an algorithm access module, and storing results of each scene. And calculating according to a weighted average algorithm by using a subsystem in the evaluation module, and outputting a final result. In particular, when subjective scoring is provided, the weighting coefficients used are: rule scoring 0.3 model scoring 0.3 subjective scoring 0.4. When subjective scoring is not available, the weighting coefficients used are: rule scoring 0.5 model scoring 0.5. And finally, evaluating the algorithm performance and displaying the algorithm performance through a visual interface.
The specific process of executing the above steps in this embodiment is described in detail in the above embodiments, and will not be repeated here.
The embodiment of the invention also provides a computer storage medium, and the computer storage medium is stored with computer program instructions; the computer program instructions, when executed by a processor, implement the method of evaluating a regulatory algorithm as described above.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
In this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a list of elements is included, and may include other elements not expressly listed.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. An evaluation method of a rule control algorithm is characterized by comprising the following steps:
the vehicle end identifies an abnormal scene according to vehicle data, wherein the vehicle data comprises external data and internal data;
after the vehicle end identifies the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the lower end of the line;
the offline end builds a virtual scene according to the abnormal scene data, and evaluates a rule control algorithm corresponding to the abnormal scene based on the virtual scene;
the rule control algorithm corresponding to the abnormal scene is evaluated based on the virtual scene, and the rule control algorithm further comprises:
evaluating a rule control algorithm in the virtual scene by adopting at least two evaluation methods of rule evaluation, model evaluation and manual evaluation;
and carrying out weighted average on evaluation values of at least two evaluation methods in rule evaluation, model evaluation and manual evaluation to obtain a third evaluation result of the rule control algorithm.
2. The method for evaluating a control algorithm according to claim 1, wherein the vehicle end identifies an abnormal scene according to vehicle data, comprising:
judging whether the vehicle data accords with preset conditions or not;
if yes, confirming that the scene corresponding to the vehicle data is an abnormal scene;
performing preliminary evaluation on the rule control algorithm of the abnormal scene to obtain a first evaluation result;
and determining whether to transmit the abnormal scene data to the offline end according to the first evaluation result.
3. The method for evaluating the rule and control algorithm according to claim 2, wherein after the vehicle end recognizes the abnormal scene, generating abnormal scene data and transmitting the abnormal scene data to the line lower end, comprising:
acquiring a data acquisition reference time point;
acquiring external data of 2n+5 minutes in the front n minutes and the rear n+5 minutes of the data acquisition reference time point;
generating abnormal scene data according to the external data, the first evaluation result and the high-precision map;
and optimally compressing the abnormal scene data, and transmitting the abnormal scene data to the offline end through road side communication.
4. The method for evaluating a rule and control algorithm according to claim 3, wherein the off-line end constructs a virtual scene according to the abnormal scene data, comprising:
cleaning the external data to obtain physical characteristics of dynamic obstacles in the abnormal scene;
inputting the cleaned external data and the high-precision map into a deep learning model to obtain a plurality of groups of abnormal scenes including dynamic obstacles under different environmental information;
and constructing a virtual scene according to the multiple groups of abnormal scenes.
5. The method for evaluating a control algorithm according to claim 4, wherein after obtaining the abnormal scene including the dynamic obstacle under the plurality of sets of different environmental information, further comprising:
screening, quantifying and grading the multiple groups of abnormal scenes;
acquiring the preset passing rate corresponding to the levels of the screened multiple groups of abnormal scenes;
and comparing the relation between the actual passing rate of the rule algorithm and the preset passing rate to obtain a second evaluation result.
6. The method for evaluating a rule and control algorithm according to claim 4 or 5, wherein after constructing a virtual scene from the plurality of sets of abnormal scenes, further comprising:
combining the multiple groups of abnormal scenes and vehicle data of the abnormal scenes to generate virtual abnormal scenes;
registering the virtual abnormal scene to a scene library.
7. The evaluation system of the rule control algorithm is characterized by comprising a vehicle end, an off-line end and a road side communication module;
the vehicle end is used for identifying an abnormal scene according to vehicle data and generating abnormal scene data, wherein the vehicle data comprises external data and internal data;
the road side communication module is used for transmitting the abnormal scene data to the lower end of the line;
the offline end is used for constructing a virtual scene according to the abnormal scene data and evaluating a rule control algorithm corresponding to the abnormal scene based on the virtual scene;
the rule control algorithm corresponding to the abnormal scene is evaluated based on the virtual scene, and the rule control algorithm further comprises:
evaluating a rule control algorithm in the virtual scene by adopting at least two evaluation methods of rule evaluation, model evaluation and manual evaluation;
and carrying out weighted average on evaluation values of at least two evaluation methods in rule evaluation, model evaluation and manual evaluation to obtain a third evaluation result of the rule control algorithm.
8. The system for evaluating a regulatory algorithm of claim 7, further comprising:
and the high-precision map module is used for storing high-precision map information of the area required by the automatic driving area and providing map information for the automatic driving and evaluating module.
9. A computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement a method of evaluating a regulatory algorithm as defined in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011262736.8A CN112462759B (en) | 2020-11-12 | 2020-11-12 | Evaluation method, system and computer storage medium of rule control algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011262736.8A CN112462759B (en) | 2020-11-12 | 2020-11-12 | Evaluation method, system and computer storage medium of rule control algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112462759A CN112462759A (en) | 2021-03-09 |
| CN112462759B true CN112462759B (en) | 2024-02-27 |
Family
ID=74825597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011262736.8A Active CN112462759B (en) | 2020-11-12 | 2020-11-12 | Evaluation method, system and computer storage medium of rule control algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112462759B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113535569B (en) * | 2021-07-22 | 2022-12-16 | 中国第一汽车股份有限公司 | Control effect determination method for automatic driving |
| CN114077218B (en) * | 2022-01-19 | 2022-04-22 | 浙江吉利控股集团有限公司 | Road data evaluation report generation method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107782564A (en) * | 2017-10-30 | 2018-03-09 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle evaluation system and method |
| CN110126846A (en) * | 2019-05-24 | 2019-08-16 | 北京百度网讯科技有限公司 | Representation method, device, system and the storage medium of Driving Scene |
| KR102011664B1 (en) * | 2018-06-07 | 2019-08-19 | 계명대학교 산학협력단 | Test evaluation apparatus for autonomous emergency braking system and control method thereof |
| US10636295B1 (en) * | 2019-01-30 | 2020-04-28 | StradVision, Inc. | Method and device for creating traffic scenario with domain adaptation on virtual driving environment for testing, validating, and training autonomous vehicle |
| CN111881520A (en) * | 2020-07-31 | 2020-11-03 | 广州文远知行科技有限公司 | Anomaly detection method and device for automatic driving test, computer equipment and storage medium |
-
2020
- 2020-11-12 CN CN202011262736.8A patent/CN112462759B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107782564A (en) * | 2017-10-30 | 2018-03-09 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle evaluation system and method |
| KR102011664B1 (en) * | 2018-06-07 | 2019-08-19 | 계명대학교 산학협력단 | Test evaluation apparatus for autonomous emergency braking system and control method thereof |
| US10636295B1 (en) * | 2019-01-30 | 2020-04-28 | StradVision, Inc. | Method and device for creating traffic scenario with domain adaptation on virtual driving environment for testing, validating, and training autonomous vehicle |
| CN110126846A (en) * | 2019-05-24 | 2019-08-16 | 北京百度网讯科技有限公司 | Representation method, device, system and the storage medium of Driving Scene |
| CN111881520A (en) * | 2020-07-31 | 2020-11-03 | 广州文远知行科技有限公司 | Anomaly detection method and device for automatic driving test, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112462759A (en) | 2021-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110164128B (en) | City-level intelligent traffic simulation system | |
| CN109782730B (en) | Method and apparatus for autonomic system performance and rating | |
| JP7345639B2 (en) | Multi-agent simulation | |
| KR20180113447A (en) | Electronic apparatus, alert message providing method of thereof and non-transitory computer readable recording medium | |
| CN112462759B (en) | Evaluation method, system and computer storage medium of rule control algorithm | |
| US20190072961A1 (en) | Autonomous driving adjustment method, apparatus, and system | |
| WO2018171875A1 (en) | Control device, system and method for determining the perceptual load of a visual and dynamic driving scene | |
| CN112215120B (en) | Method and device for determining visual search area and driving simulator | |
| JP2009096365A (en) | Risk recognition system | |
| CN110696835A (en) | Automatic early warning method and automatic early warning system for dangerous driving behaviors of vehicle | |
| CN117057656A (en) | Digital twinning-based smart city management method and system | |
| CN113642114A (en) | Modeling method for humanoid random car following driving behavior capable of making mistakes | |
| CN112633074A (en) | Pedestrian information detection method and device, storage medium and electronic equipment | |
| CN115761551A (en) | Automatic driving simulation scene generation method, device, equipment and medium | |
| CN115416673A (en) | Driving state detection device and method, electronic equipment and vehicle | |
| CN115169477A (en) | Method, device and equipment for evaluating safety sense of assistant driving system and storage medium | |
| Hammit et al. | Radar-vision algorithms to process the trajectory-level driving data in the SHRP2 Naturalistic Driving Study | |
| Xie et al. | Deepcf: A deep feature learning-based car-following model using online ride-hailing trajectory data | |
| Luo et al. | Dynamic simplex: Balancing safety and performance in autonomous cyber physical systems | |
| Ma et al. | Lane change analysis and prediction using mean impact value method and logistic regression model | |
| Tang | Application of information entropy algorithm in safety risk prediction of road traffic driving behaviors. | |
| US20230195977A1 (en) | Method and system for classifying scenarios of a virtual test, and training method | |
| CN116125996B (en) | Safety monitoring method and system for unmanned vehicle | |
| Husen et al. | Continuous Car Driving intention recognition with syntactic pattern approach | |
| CN116776204B (en) | Driver risk sensitivity differentiation characterization method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |