CN114896754B - Automatic driving system performance evaluation method oriented to logic scene full parameter space - Google Patents
Automatic driving system performance evaluation method oriented to logic scene full parameter space Download PDFInfo
- Publication number
- CN114896754B CN114896754B CN202210285067.9A CN202210285067A CN114896754B CN 114896754 B CN114896754 B CN 114896754B CN 202210285067 A CN202210285067 A CN 202210285067A CN 114896754 B CN114896754 B CN 114896754B
- Authority
- CN
- China
- Prior art keywords
- scene
- vehicle
- evaluation
- collision
- parameter space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 90
- 238000012360 testing method Methods 0.000 claims abstract description 38
- 238000000034 method Methods 0.000 claims description 51
- 238000005070 sampling Methods 0.000 claims description 48
- 238000004422 calculation algorithm Methods 0.000 claims description 42
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000012937 correction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 239000002245 particle Substances 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000013316 zoning Methods 0.000 claims description 3
- 238000005192 partition Methods 0.000 abstract description 3
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/40—Business processes related to the transportation industry
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of automatic driving automobile test evaluation, and particularly relates to an automatic driving automobile performance evaluation method oriented to a logic scene full parameter space. According to the performance evaluation method, after a test logic scene of an automatic driving system and a matched parameter space are given, the tested automatic driving system is put into the logic scene for testing, running data under each specific test working condition are obtained, after a running track field of the tested automatic driving system in the whole tested logic scene parameter space is obtained, the logic scene is partitioned into a safety area and a dangerous area according to an ideal vehicle motion curve; then determining evaluation emphasis and evaluation indexes in the two partitions; and calculating the overall performance in the whole parameter space according to the test result, thereby obtaining the performance evaluation result of the whole parameter space.
Description
Technical Field
The invention belongs to the technical field of automatic driving automobile test evaluation, and particularly relates to an automatic driving automobile performance evaluation method oriented to a logic scene full parameter space.
Background
With the continuous improvement of the technology of the automatic driving system, each vehicle enterprise pushes out its own automatic driving vehicle type, however, how to evaluate the performance level of each automatic driving vehicle is still not unified standard. At present, a scene-based test method has become a mainstream of safety verification of an automatic driving system, and a test scene is determined according to various working conditions possibly encountered in the driving process of the automatic driving system, wherein a logic scene describes the same type of test working conditions by using a parameter space form, and is a main level of the existing scene-based test evaluation method.
Most of the evaluation methods of the existing methods aim at evaluating single parameter working conditions, and evaluate specific scenes at different positions in a logic scene parameter space by using the same evaluation dimension, and lack a performance evaluation method for the logic scene full parameter space.
Disclosure of Invention
The invention provides an automatic driving automobile performance evaluation method facing to a logic scene full parameter space, after a test logic scene of a tested automatic driving system and a parameter space matched with the test logic scene are given, the tested automatic driving system is put into the logic scene for testing, running data under each specific test working condition are obtained, after a running track field of the tested automatic driving system in the whole tested logic scene parameter space is obtained, the logic scene is partitioned into a safety area and a danger area according to an ideal vehicle motion curve; then determining evaluation emphasis and evaluation indexes in the two partitions; and calculating the overall performance in the whole parameter space according to the test result, thereby obtaining the performance evaluation result of the whole parameter space.
The technical scheme of the invention is as follows in combination with the accompanying drawings:
a logic scene full parameter space-oriented automatic driving automobile performance evaluation method comprises the following steps:
firstly, partitioning a logic scene parameter space;
Step two, determining evaluation dimensions in different areas of the logic scene parameter space;
Step three, calculating a driving track field;
Step four, constructing a smoothness evaluation index in the safety zone;
Step five, constructing collision avoidance evaluation indexes in the dangerous area;
And step six, constructing a logic scene full-parameter space evaluation index.
The specific method of the first step is as follows:
Dividing a logic scene parameter space into a dangerous area and a safe area according to natural driving data of a human driver or preset dangerous conditions in the running process of a qualified system; the dangerous area is an area where collision is likely to happen during driving; the safety area is an area which is safe in driving and is difficult to collide;
The principle of zoning is that the vehicle only carries out deceleration operation, and whether danger can be avoided or not; the maximum braking deceleration is set according to the road adhesion coefficient of a test scene, various information of a vehicle carrying a qualified system or a human driver, which can timely and accurately sense a front obstacle, is included in the information, wherein the information comprises types, speeds and positions, when the vehicle runs to a dangerous distance set by a safe distance model, the vehicle immediately takes a deceleration operation, the information of the inverse collision time of the vehicle carrying the qualified system in the whole running scene, namely ITTC, is recorded, and the calculation formula of the inverse collision time is shown as a formula (1); if the maximum value of ITTC in the whole driving mileage is not more than 0.7s -1, the scene is considered as a safety scene, otherwise, the scene is considered as a dangerous scene;
ITTC＝v/d (1)
Wherein d is the relative distance between the vehicle to be tested and the obstacle; v is the relative speed between the vehicle under test and the obstacle;
Because all parameter combinations in the parameter space cannot be tested, the accurate continuous boundary between the dangerous area and the safe area cannot be obtained by a testing method; fitting the boundary by using a Gaussian process; the gaussian process is shown in formula (2):
f(x)～GP(m,k) (2)
Wherein f (x) is a fitting result; m is a mean function; k is a covariance function; m is defined as a 0 matrix, and the kernel density function selects a square-exponential kernel function as shown in formula (3):
Wherein σ f is a feature length scalar; σ l is the signal standard deviation; x is training data; x * is data at an unknown location;
And finally, dividing the dangerous part and the safe part in the parameter space through Gaussian process fitting, thereby obtaining a dangerous area and a safe area.
The specific method of the second step is as follows:
The evaluation dimension in the dangerous area is embodied by collision avoidance indexes; the evaluation dimension in the safety zone is embodied by a smoothness index;
the specific method of the third step is as follows:
The driving track field refers to a track remained by the influence on the surrounding space in the driving process of the vehicle, is related to the driving track, the corresponding position speed, the corresponding position influence time and the vehicle physical parameters of the vehicle, and is aimed at the same vehicle in the smoothness evaluation, so that the influence of the vehicle physical parameters is removed; the defined track field is shown in a formula (4), and when the vehicle is stopped after operation, the average duration of the time passing through the scene is calculated; the ambient space influence calculation for a specific point in time is shown in equation (5);
S＝∑s (4)
wherein S is a running track field, namely the sum of influences of the whole running process of the vehicle on surrounding space; s is an instantaneous field, namely the influence of the moment of vehicle running on the surrounding space time; r ij is a vector formed by different positions and the center of the vehicle; v i is the speed of the vehicle; θ i is the included angle between r ij and v i; k 1 and k 2 are correction parameters; the vehicle is considered a particle and the instantaneous field value of the vehicle heading away from the vehicle centroid 1m is considered the instantaneous field value throughout the vehicle 1 m.
The specific method of the fourth step is as follows:
Evaluating the compliance of the algorithm to be tested by using the similarity between the algorithm to be tested and natural driving data of a human driver or a preset track field of an ideal system; after the track field of the estimated algorithm and the ideal system in the same test scene is calculated, firstly selecting a sampling line segment every 5m according to the road length, and selecting sampling points on the sampling line segment; sampling points which are 0.5m away from the driving center are selected when sampling points are selected, namely, firstly, sampling is carried out forwards along the road direction, the sampling interval is 5m, in one sampling line segment, sampling is carried out with the vehicle center point as the center and the upper and lower intervals of 0.5m, 4 sampling points are respectively sampled up and down, 9 sampling points are added to the driving center sampling points, and the similarity of a measured algorithm and a rational algorithm at the sampling point positions is equal to
dij＝1-|hij1-hij2|/(hij2+hij1) (6)
Wherein h ij1 is the running track field value of the tested automatic driving system at the sampling point; h ij2 is the running track field value of the sampling point processing algorithm;
Sampling to obtain all d ij on the whole road length, wherein the smoothness of the running process of the estimated algorithm and the ideal algorithm in a specific scene is that
Under the real natural driving condition, the higher the probability of occurrence of one scene, the longer the time that a driver experiences the scene, and the larger the proportion of the scene, so that after the smoothness evaluation index of a single specific scene is obtained, the probability is taken as the weight to obtain the evaluation result in the whole safety zone, as shown in (8)
Wherein p i is the occurrence probability of the ith specific scene in the scene of the type under the natural driving condition.
The specific method of the fifth step is as follows:
When the collision avoidance performance in the dangerous area is evaluated, the evaluation content comprises two parts, wherein one part is the evaluation content for avoiding the danger, and the other part is the evaluation content for reducing the collision loss when the danger is unavoidable; firstly, defining the concept of collision loss;
Wherein L i is the collision loss of the estimated algorithm in the ith specific scene; w is the utilization rate of the bumper of the vehicle to be tested in the collision process, namely the area of the bumper participating in the collision, and the minimum is 0.5; v e and v o are the speed of the vehicle under test at the time of collision and the speed of the collision obstacle at the time of collision; u i is the crash severity;
In addition to collision loss, the importance weights of scenes at different locations are also different; for a collision, once it occurs, it is a certain event for the occupant, thus taking into account the collision avoidance capabilities in the danger zone, rather than the relative probability of occurrence of the relevant scene;
In the evaluation of the collision avoidance capability, the calculation method of the distance is shown as formula (11) taking the relative distance between the current specific scene parameter position and the most dangerous parameter position into consideration
Where r i is the relative weight of the ith specific scene in the hazard zone; r i * is a vector formed by the most dangerous parameter points and specific scene parameter points in the logic scene parameter space; r i ** is a vector formed by the intersection of the most dangerous parameter point in the logical scene parameter space and the straight line where r i * is located and the boundary of the safety area/danger area;
after the collision loss and the corresponding weight of the estimated algorithm in the specific scene are obtained, the evaluation index in the whole dangerous area is obtained; since the ideal algorithm assumes that the sensing system, the decision system and the execution system all operate in an ideal state in the operation process, the collision loss is necessarily minimum, and therefore the value of C is necessarily less than or equal to 1.
Wherein, C is a safety index; l gi is the collision loss of the ideal system in the ith specific scenario in the hazard zone; n c is the number of specific scenes in which all collisions occur.
The specific method in the step six is as follows:
After the evaluation index in the safety area and the evaluation index in the danger area are obtained, combining the evaluation index and the evaluation index to obtain the performance evaluation of the estimated algorithm in the whole logic scene parameter space; based on the percentage, the comprehensive evaluation index result of the measured algorithm is
S＝a·k5·C+b·k6·D (13)
Wherein a and b are relative scores of a safety index and a smoothness index, and the sum of the safety index and the smoothness index is 100 in the case of percentage; k 5 and k 6 are correction parameters, when the number of the tested systems is lower than the threshold value, the threshold value is 10, and can be 1 respectively, and when the number of the tested systems exceeds the threshold value, the values of the two are corrected according to the statistical result, so that the evaluation results of different test systems have Gaussian distribution or exponential distribution statistical characteristics.
The beneficial effects of the invention are as follows:
According to the method, the logic scene parameter space is partitioned according to the dangers of different specific scenes, and the evaluation dimensions and specific evaluation indexes in the two partitions are determined, so that the overall performance in the whole logic scene parameter space is obtained according to the test result of the tested algorithm. Finally, the automatic driving performance evaluation of the logic scene level is obtained, and the evaluation can be carried out according to the evaluation emphasis pertinence of different specific scene positions.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a method for evaluating performance of an automatic driving automobile facing to a full parameter space of a logic scene includes the following steps:
firstly, partitioning a logic scene parameter space;
the specific method comprises the following steps:
Dividing a logic scene parameter space into a dangerous area and a safe area according to natural driving data of a human driver or preset dangerous conditions in the running process of a qualified system; the dangerous area is an area where collision is likely to happen during driving; the safety area is an area which is safe in driving and is difficult to collide;
The principle of zoning is that the vehicle only carries out deceleration operation, and whether danger can be avoided or not; the maximum braking deceleration is set according to the road adhesion coefficient of a test scene, various information of a vehicle carrying a qualified system or a human driver, which can timely and accurately sense a front obstacle, is included in the information, wherein the information comprises types, speeds and positions, when the vehicle runs to a dangerous distance set by a safe distance model, the vehicle immediately takes a deceleration operation, the information of the inverse collision time of the vehicle carrying the qualified system in the whole running scene, namely ITTC, is recorded, and the calculation formula of the inverse collision time is shown as a formula (1); if the maximum value of ITTC in the whole driving mileage is not more than 0.7s -1, the scene is considered as a safety scene, otherwise, the scene is considered as a dangerous scene;
ITTC＝v/d (1)
Wherein d is the relative distance between the vehicle to be tested and the obstacle; v is the relative speed between the vehicle under test and the obstacle;
Because all parameter combinations in the parameter space cannot be tested, the accurate continuous boundary between the dangerous area and the safe area cannot be obtained by a testing method; fitting the boundary by using a Gaussian process; the gaussian process is shown in formula (2):
f(x)～GP(m,k) (2)
Wherein f (x) is a fitting result; m is a mean function; k is a covariance function; m is defined as a 0 matrix, and the kernel density function selects a square-exponential kernel function as shown in formula (3):
Wherein σ f is a feature length scalar; σ l is the signal standard deviation; x is training data; x * is data at an unknown location;
And finally, dividing the dangerous part and the safe part in the parameter space through Gaussian process fitting, thereby obtaining a dangerous area and a safe area.
Step two, determining evaluation dimensions in different areas of the logic scene parameter space;
The evaluation dimension in the dangerous area is embodied by collision avoidance indexes; the evaluation dimension in the safety zone is embodied by a smoothness index.
The method comprises the following steps:
Performing discrete processing on parameter spaces in the safety area and the dangerous area to obtain a specific scene set for testing, and performing simulation test on the estimated algorithm by using the specific scenes to obtain a test result; when the smoothness index evaluation is carried out on the safety zone, if the test result has collision, no smoothness index calculation is carried out in the scene of the safety zone, the collision loss is calculated on the collision result in the safety zone through the loss model in the step five, and the scene parameters at the collision position and the test result are placed in a collision set; when no collision occurs in the whole space of the safety zone, calculating a single specific scene smoothness index in each specific scene and a smoothness index in the whole safety zone according to the fourth step; calculating collision avoidance indexes of the measured algorithm in the dangerous area according to the fifth step; and step six, combining the safety index and the smoothness index to obtain a comprehensive evaluation result of the whole parameter space of the logic scene.
Step three, calculating a driving track field;
the specific method comprises the following steps:
The driving track field refers to a track remained by the influence on the surrounding space in the driving process of the vehicle, is related to the driving track, the corresponding position speed, the corresponding position influence time and the vehicle physical parameters of the vehicle, and is aimed at the same vehicle in the smoothness evaluation, so that the influence of the vehicle physical parameters is removed; the defined track field is shown in a formula (4), and when the vehicle is stopped after operation, the average duration of the time passing through the scene is calculated; the ambient space influence calculation for a specific point in time is shown in equation (5);
S＝∑s (4)
wherein S is a running track field, namely the sum of influences of the whole running process of the vehicle on surrounding space; s is an instantaneous field, namely the influence of the moment of vehicle running on the surrounding space time; r ij is a vector formed by different positions and the center of the vehicle; v i is the speed of the vehicle; θ i is the included angle between r ij and v i; k 1 and k 2 are correction parameters; the vehicle is considered a particle and the instantaneous field value of the vehicle heading away from the vehicle centroid 1m is considered the instantaneous field value throughout the vehicle 1 m.
Step four, constructing a smoothness evaluation index in the safety zone;
the specific method comprises the following steps:
Evaluating the compliance of the algorithm to be tested by using the similarity between the algorithm to be tested and natural driving data of a human driver or a preset track field of an ideal system; after the track field of the estimated algorithm and the ideal system in the same test scene is calculated, firstly selecting a sampling line segment every 5m according to the road length, and selecting sampling points on the sampling line segment; sampling points which are 0.5m away from the driving center are selected when sampling points are selected, namely, firstly, sampling is carried out forwards along the road direction, the sampling interval is 5m, in one sampling line segment, sampling is carried out with the vehicle center point as the center and the upper and lower intervals of 0.5m, 4 sampling points are respectively sampled up and down, 9 sampling points are added to the driving center sampling points, and the similarity of a measured algorithm and a rational algorithm at the sampling point positions is equal to
dij＝1-|hij1-hij2|/(hij2+hij1) (6)
Wherein h ij1 is the running track field value of the tested automatic driving system at the sampling point; h ij2 is the running track field value of the sampling point processing algorithm;
Sampling to obtain all d ij on the whole road length, wherein the smoothness of the running process of the estimated algorithm and the ideal algorithm in a specific scene is that
Under the real natural driving condition, the higher the probability of occurrence of one scene, the longer the time that a driver experiences the scene, and the larger the proportion of the scene, so that after the smoothness evaluation index of a single specific scene is obtained, the probability is taken as the weight to obtain the evaluation result in the whole safety zone, as shown in (8)
Wherein p i is the occurrence probability of the ith specific scene in the scene of the type under the natural driving condition.
Step five, constructing collision avoidance evaluation indexes in the dangerous area;
the specific method comprises the following steps:
When the collision avoidance performance in the dangerous area is evaluated, the evaluation content comprises two parts, wherein one part is the evaluation content for avoiding the danger, and the other part is the evaluation content for reducing the collision loss when the danger is unavoidable; firstly, defining the concept of collision loss;
Wherein L i is the collision loss of the estimated algorithm in the ith specific scene; w is the utilization rate of the bumper of the vehicle to be tested in the collision process, namely the area of the bumper participating in the collision, and the minimum is 0.5; v e and v o are the speed of the vehicle under test at the time of collision and the speed of the collision obstacle at the time of collision; u i is the crash severity;
In addition to collision loss, the importance weights of scenes at different locations are also different; for a collision, once it occurs, it is a certain event for the occupant, thus taking into account the collision avoidance capabilities in the danger zone, rather than the relative probability of occurrence of the relevant scene;
In the evaluation of the collision avoidance capability, the calculation method of the distance is shown as formula (11) taking the relative distance between the current specific scene parameter position and the most dangerous parameter position into consideration
Where r i is the relative weight of the ith specific scene in the hazard zone; r i * is a vector formed by the most dangerous parameter points and specific scene parameter points in the logic scene parameter space; r i ** is a vector formed by the intersection of the most dangerous parameter point in the logical scene parameter space and the straight line where r i * is located and the boundary of the safety area/danger area;
after the collision loss and the corresponding weight of the estimated algorithm in the specific scene are obtained, the evaluation index in the whole dangerous area is obtained; since the ideal algorithm assumes that the sensing system, the decision system and the execution system all operate in an ideal state in the operation process, the collision loss is necessarily minimum, and therefore the value of C is necessarily less than or equal to 1.
Wherein, C is a safety index; l gi is the collision loss of the ideal system in the ith specific scenario in the hazard zone; n c is the number of specific scenes in which all collisions occur.
And step six, constructing a logic scene full-parameter space evaluation index.
The specific method comprises the following steps:
After the evaluation index in the safety area and the evaluation index in the danger area are obtained, combining the evaluation index and the evaluation index to obtain the performance evaluation of the estimated algorithm in the whole logic scene parameter space; based on the percentage, the comprehensive evaluation index result of the measured algorithm is
S＝a·k5·C+b·k6·D (13)
Wherein a and b are relative scores of a safety index and a smoothness index, and the sum of the safety index and the smoothness index is 100 in the case of percentage; k 5 and k 6 are correction parameters, when the number of the tested systems is lower than the threshold value, the threshold value is 10, and can be 1 respectively, and when the number of the tested systems exceeds the threshold value, the values of the two are corrected according to the statistical result, so that the evaluation results of different test systems have Gaussian distribution or exponential distribution statistical characteristics.
The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the scope of the present invention is not limited to the specific details of the above embodiments, and within the scope of the technical spirit of the present invention, any person skilled in the art may apply equivalent substitutions or alterations to the technical solution of the present invention and the inventive concept thereof within the scope of the technical spirit of the present invention, and these simple modifications are all within the scope of the present invention.
In addition, the specific features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various possible combinations are not described further.
Moreover, any combination of the various embodiments of the invention can be made without departing from the spirit of the invention, which should also be considered as disclosed herein.
Claims (2)
1. A logic scene full parameter space-oriented automatic driving automobile performance evaluation method is characterized by comprising the following steps:
firstly, partitioning a logic scene parameter space;
Step two, determining evaluation dimensions in different areas of the logic scene parameter space;
Step three, calculating a driving track field;
Step four, constructing a smoothness evaluation index in the safety zone;
Step five, constructing collision avoidance evaluation indexes in the dangerous area;
Step six, constructing a logic scene full-parameter space evaluation index;
The specific method of the first step is as follows:
Dividing a logic scene parameter space into a dangerous area and a safe area according to natural driving data of a human driver or preset dangerous conditions in the running process of a qualified system; the dangerous area is an area where collision is likely to happen during driving; the safety area is an area which is safe in driving and is difficult to collide;
The principle of zoning is that the vehicle only carries out deceleration operation, and whether danger can be avoided or not; the maximum braking deceleration is set according to the road surface adhesion coefficient of the test scene, and in the running process, a vehicle carrying a qualified system or a human driver can timely and accurately sense various information of a front obstacle, wherein the information comprises the type, the speed and the position; when the vehicle runs to the dangerous distance set by the safe distance model, the vehicle immediately adopts a deceleration operation, the information of the inverse collision time of the vehicle carrying the qualified system in the whole running scene, namely ITTC, is recorded, and the calculation formula of the inverse collision time is shown in a formula (1); if the maximum value of ITTC in the whole driving mileage is not more than 0.7s -1, the scene is considered as a safety scene, otherwise, the scene is considered as a dangerous scene;
ITTC ＝ v / d (1)
Wherein d is the relative distance between the vehicle to be tested and the obstacle; v is the relative speed between the vehicle under test and the obstacle;
Because all parameter combinations in the parameter space cannot be tested, the accurate continuous boundary between the dangerous area and the safe area cannot be obtained by a testing method; fitting the boundary by using a Gaussian process; the gaussian process is shown in formula (2):
f(x)～GP(m,k)(2)
Wherein f (x) is a fitting result; m is a mean function; k is a covariance function; m is defined as a 0 matrix, and the kernel density function selects a square-exponential kernel function as shown in formula (3):
Wherein σ f is a feature length scalar; σ l is the signal standard deviation; x is training data; x * is data at an unknown location;
Fitting through a Gaussian process, and finally dividing a dangerous part and a safe part in a parameter space to obtain a dangerous area and a safe area;
the specific method of the third step is as follows:
The driving track field refers to a track remained by the influence on the surrounding space in the driving process of the vehicle, is related to the driving track, the corresponding position speed, the corresponding position influence time and the vehicle physical parameters of the vehicle, and is aimed at the same vehicle in the smoothness evaluation, so that the influence of the vehicle physical parameters is removed; the defined track field is shown in a formula (4), and when the vehicle is stopped after operation, the average duration of the time passing through the scene is calculated; the ambient space influence calculation for a specific point in time is shown in equation (5);
S＝Σs (4)
Wherein S is a running track field, namely the sum of influences of the whole running process of the vehicle on surrounding space; s is an instantaneous field, namely the influence of the moment of vehicle running on the surrounding space time; r ij is a vector formed by different positions and the center of the vehicle; v i is the speed of the vehicle; θ i is the included angle between r ij and v i; k 1 and k 2 are correction parameters; considering the vehicle as a particle and considering the instantaneous field value of the vehicle in the forward direction, which is far from the vehicle centroid 1m, as the instantaneous field value in the whole vehicle 1 m;
the specific method of the fourth step is as follows:
Evaluating the compliance of the algorithm to be tested by using the similarity between the algorithm to be tested and natural driving data of a human driver or a preset track field of an ideal system; after the track field of the estimated algorithm and the ideal system in the same test scene is calculated, firstly selecting a sampling line segment every 5m according to the road length, and selecting sampling points on the sampling line segment; sampling points which are 0.5m away from the driving center are selected when sampling points are selected, namely, firstly, sampling is carried out forwards along the road direction, the sampling interval is 5m, in one sampling line segment, sampling is carried out with the vehicle center point as the center and the upper and lower intervals of 0.5m, 4 sampling points are respectively sampled up and down, 9 sampling points are added to the driving center sampling points, and the similarity of a measured algorithm and a rational algorithm at the sampling point positions is equal to
dij＝1-|hij1-hij2|/(hij2+hij1) (6)
Wherein h ij1 is the running track field value of the tested automatic driving system at the sampling point; h ij2 is the value of the driving track field of the human driver or the processing algorithm of the sampling point;
sampling to obtain all d ij of the whole road length, wherein the universality of the estimated algorithm in the whole driving process is that
Where n is the number of samples of d ij;
Under the real natural driving condition, the higher the probability of occurrence of one scene, the longer the time that a driver experiences the scene, and the larger the proportion of the scene, so that after the smoothness evaluation index of a single specific scene is obtained, the probability is taken as the weight to obtain the evaluation result in the whole safety zone, as shown in (8)
Wherein, p i is the occurrence probability of the ith specific scene in the scene of the type under the natural driving condition;
The specific method of the fifth step is as follows:
When the collision avoidance performance in the dangerous area is evaluated, the evaluation content comprises two parts, wherein one part is the evaluation content for avoiding the danger, and the other part is the evaluation content for reducing the collision loss when the danger is unavoidable; firstly, defining the concept of collision loss;
Wherein L i is the collision loss of the estimated algorithm in the ith specific scene; w is the utilization rate of the bumper of the vehicle to be tested in the collision process, namely the area of the bumper participating in the collision, and the minimum is 0.5; v e and v o are the speed of the vehicle under test at the time of collision and the speed of the collision obstacle at the time of collision; u i is the crash severity;
In addition to collision loss, the importance weights of scenes at different locations are also different; for a collision, once it occurs, it is a certain event for the occupant, thus taking into account the collision avoidance capabilities in the danger zone, rather than the relative probability of occurrence of the relevant scene;
In the evaluation of the collision avoidance capability, the calculation method of the distance is shown as formula (11) taking the relative distance between the current specific scene parameter position and the most dangerous parameter position into consideration
Where r i is the relative weight of the ith specific scene in the hazard zone; r i * is a vector formed by the most dangerous parameter points and specific scene parameter points in the logic scene parameter space; r i ** is a vector formed by the intersection of the most dangerous parameter point in the logical scene parameter space and the straight line where r i * is located and the boundary of the safety area/danger area;
After the collision loss and the corresponding weight of the estimated algorithm in the specific scene are obtained, the evaluation index in the whole dangerous area is obtained; because the ideal algorithm assumes that the sensing system, the decision system and the execution system all operate in an ideal state in the operation process, the collision loss is always minimum, and the value of C is always less than or equal to 1;
Wherein, C is a safety index; l gi is the collision loss of the ideal system in the ith specific scenario in the hazard zone; n c is the number of specific scenes in which all collisions occur; l i is the collision loss of the estimated algorithm in the ith specific scenario;
The specific method in the step six is as follows:
After the evaluation index in the safety area and the evaluation index in the danger area are obtained, combining the evaluation index and the evaluation index to obtain the performance evaluation of the estimated algorithm in the whole logic scene parameter space; based on the percentage, the comprehensive evaluation index result of the measured algorithm is
S＝a·k5·C+b·k6·D (13)
Wherein a and b are relative scores of a safety index and a smoothness index, and the sum of the safety index and the smoothness index is 100 in the case of percentage; k 5 and k 6 are correction parameters, when the number of the tested systems is lower than the threshold value, the threshold value is 10, and when the number of the tested systems exceeds the threshold value, the values of the two are corrected according to the statistical result, so that the evaluation results of different test systems have Gaussian distribution or exponential distribution statistical characteristics.
2. The method for evaluating the performance of the automatic driving automobile facing the full parameter space of the logic scene according to claim 1, wherein the specific method of the second step is as follows:
The evaluation dimension in the dangerous area is embodied by collision avoidance indexes; the evaluation dimension in the safety zone is embodied by a smoothness index.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285067.9A CN114896754B (en) | 2022-03-22 | 2022-03-22 | Automatic driving system performance evaluation method oriented to logic scene full parameter space |
| US17/943,167 US20230304896A1 (en) | 2022-03-22 | 2022-09-12 | Method for evaluating performance of self-driving vehicle oriented to full parameter space of logical scenario |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285067.9A CN114896754B (en) | 2022-03-22 | 2022-03-22 | Automatic driving system performance evaluation method oriented to logic scene full parameter space |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114896754A CN114896754A (en) | 2022-08-12 |
| CN114896754B true CN114896754B (en) | 2024-06-04 |
Family
ID=82715577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210285067.9A Active CN114896754B (en) | 2022-03-22 | 2022-03-22 | Automatic driving system performance evaluation method oriented to logic scene full parameter space |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114896754B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117698770B (en) * | 2024-02-06 | 2024-04-12 | 北京航空航天大学 | Automatic driving decision safety collision risk assessment method based on multi-scene fusion |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019215599A (en) * | 2018-06-11 | 2019-12-19 | アイシン・エィ・ダブリュ株式会社 | Operation evaluation device and operation evaluation program |
| US10831207B1 (en) * | 2018-08-22 | 2020-11-10 | BlueOwl, LLC | System and method for evaluating the performance of a vehicle operated by a driving automation system |
| CN113419942A (en) * | 2021-04-30 | 2021-09-21 | 吉林大学 | Automatic driving safety evaluation method based on natural driving data |
| CN113985875A (en) * | 2021-10-27 | 2022-01-28 | 福州大学 | Artificial potential field unmanned vehicle dynamic path planning method based on collision prediction model |
| CN114021327A (en) * | 2021-10-28 | 2022-02-08 | 同济大学 | Quantitative evaluation method for performance of automatic driving automobile sensing system |
-
2022
- 2022-03-22 CN CN202210285067.9A patent/CN114896754B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019215599A (en) * | 2018-06-11 | 2019-12-19 | アイシン・エィ・ダブリュ株式会社 | Operation evaluation device and operation evaluation program |
| US10831207B1 (en) * | 2018-08-22 | 2020-11-10 | BlueOwl, LLC | System and method for evaluating the performance of a vehicle operated by a driving automation system |
| CN113419942A (en) * | 2021-04-30 | 2021-09-21 | 吉林大学 | Automatic driving safety evaluation method based on natural driving data |
| CN113985875A (en) * | 2021-10-27 | 2022-01-28 | 福州大学 | Artificial potential field unmanned vehicle dynamic path planning method based on collision prediction model |
| CN114021327A (en) * | 2021-10-28 | 2022-02-08 | 同济大学 | Quantitative evaluation method for performance of automatic driving automobile sensing system |
Non-Patent Citations (1)
| Title |
|---|
| 面向多维度逻辑场景的自动驾驶安全性聚类评价方法;朱冰等;《汽车工程》;20201125;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114896754A (en) | 2022-08-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109871599B (en) | Unmanned vehicle operation safety index evaluation method | |
| CN110949398B (en) | Method for detecting abnormal driving behavior of first-vehicle drivers in vehicle formation driving | |
| Deng et al. | Improved driving behaviors prediction based on fuzzy logic-hidden markov model (fl-hmm) | |
| US20230304896A1 (en) | Method for evaluating performance of self-driving vehicle oriented to full parameter space of logical scenario | |
| CN114896754B (en) | Automatic driving system performance evaluation method oriented to logic scene full parameter space | |
| CN111243338A (en) | Vehicle acceleration-based collision risk evaluation method | |
| CN113658451B (en) | Vehicle collision early warning control method and device, vehicle and readable storage medium | |
| CN111582586A (en) | Multi-fleet driving risk prediction system and method for reducing jitter | |
| CN112365706B (en) | Abnormal vehicle distance identification method based on multi-source information | |
| CN112183947A (en) | Rail transit vehicle comprehensive evaluation method and device, electronic equipment and storage medium | |
| CN116135640A (en) | Anti-collision early warning method and system for vehicle and vehicle | |
| CN114090404B (en) | Automatic driving acceleration test method considering efficiency and coverage | |
| JP5750179B1 (en) | Evaluation support program and evaluation support device for automatic brake system | |
| CN110843767A (en) | Spline theory-based automatic parking method and system | |
| CN117809458A (en) | Real-time assessment method and system for traffic accident risk | |
| CN112172829B (en) | Lane departure warning method and device, electronic equipment and storage medium | |
| CN113592221A (en) | Road section risk dynamic evaluation method based on safety substitution evaluation index | |
| CN117235656A (en) | Urban traffic management system and method based on big data and cloud computing | |
| CN110047167A (en) | Vehicle zig zag number equivalent statistics method | |
| CN114996116A (en) | Anthropomorphic evaluation method for automatic driving system | |
| CN113722982A (en) | Automobile sensor attack detection and defense method based on multi-model fusion | |
| CN115731261B (en) | Vehicle lane change behavior recognition method and system based on expressway radar data | |
| CN117125083B (en) | Vehicle following behavior risk quantification method considering driving style inclination | |
| CN117104255B (en) | Intelligent perception human-vehicle interaction system and method for intelligent driving vehicle | |
| CN117719524B (en) | Driving safety risk identification early warning method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |