CN110553853A - automatic driving function test and evaluation method based on poor scene search under field - Google Patents
automatic driving function test and evaluation method based on poor scene search under field Download PDFInfo
- Publication number
- CN110553853A CN110553853A CN201910720260.9A CN201910720260A CN110553853A CN 110553853 A CN110553853 A CN 110553853A CN 201910720260 A CN201910720260 A CN 201910720260A CN 110553853 A CN110553853 A CN 110553853A
- Authority
- CN
- China
- Prior art keywords
- test
- scene
- automatic driving
- tested
- driving function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M17/00—Testing of vehicles
- G01M17/007—Wheeled or endless-tracked vehicles
Abstract
The invention discloses an automatic driving function test and evaluation method based on poor scene search under a field, which comprises the following steps: s1: analyzing an application scene of an automatic driving function; s2: determining the test site condition; s3: parameterizing a test scene; s4: defining the scene quality; s5: analyzing and determining an initial high-efficiency clustering test scene; s6: and (4) performing automatic driving function test based on poor scene search in a test scene. The invention tests the automatic driving function to be tested by applying an actual test field, so that the test conclusion is closer to the real condition; the automatic driving function test is carried out on the basis of continuously searching poor test scenes, and improvement suggestions can be directly provided for the perfection degree of the automatic driving function.
Description
Technical Field
The invention relates to the technical field of intelligent transportation, in particular to an automatic driving function testing and evaluating method based on worst scene search.
Background
how to enable the automatic driving technology to safely land is a problem which needs to be solved urgently at present, and solving the problem not only needs to further perfect the automatic driving vehicle technology but also needs to develop a proper automatic driving test and evaluation technology. The existing evaluation method related to the automatic driving function can be mainly divided into a simulation test, a hardware-in-loop test, a site test and an actual road test, because in the simulation test and the hardware-in-loop test, software is used for replacing all hardware or part of hardware of a vehicle, the reliability of the method is difficult to reach the level of the site test and the actual road test, because real road participants exist in the actual road test, dangerous tests cannot be carried out, and the site test can carry out the dangerous tests under the controllable condition, so the method is an indispensable method for ensuring the safety of the automatic driving function and the actual performance test of the automatic driving function on the basis. However, the scenes faced by the automatic driving function are complex, and the scenes have more variable parameters, so how to complete the safety and performance detection of the automatic driving function under a reasonable field test quantity is a theoretical and engineering problem which needs to be solved urgently.
disclosure of Invention
Therefore, the invention provides an automatic driving function test and evaluation method based on poor scene search in a field, which comprises the following steps:
s1: analyzing an application scene required by the performance of the automatic driving function to be tested;
s2: determining a test site condition based on application scene analysis;
S3: parameterization of a test scene under the determined test site condition comprises static scene parameterization and dynamic road participant parameterization;
S4: defining scene quality based on the automatic driving function performance to be tested and the test scene parameterization;
s5: analyzing and determining an initial high-efficiency clustering test scene based on the automatic driving function performance to be tested and the parameterization of the test scene, wherein the analysis and determination comprise the determination of the number of road participants and the determination of the initial relative position of the road participants;
S6: and the automatic driving function test based on poor scene search under the scene quality definition and the initial high-efficiency clustering test scene.
and analyzing application scenes required by the performance of the automatic driving function to be tested, including analysis of self state parameters of the automatic driving vehicle to be tested, analysis of road traffic environment of a test site, analysis of other road participants except the automatic driving vehicle to be tested, and weather conditions. And simultaneously determining auxiliary test facilities such as a road traffic environment, a soft target guide vehicle and the like which can be provided by a test site.
And defining the quality of a specific test scene on the basis of scene parameterization, and calculating and deducing the quality value of the specific test scene through the test data in each test according to the definition of the quality of the test scene. The advantages and disadvantages of the automatic driving function in one site test can be quantified through the scene advantages and disadvantages, the definition of the scene advantages and disadvantages comprises how to deduce the quantified scene advantages and disadvantages through the data calculation of a specific one-time test, and therefore comparison of different test scenes can be conducted, the worse the automatic driving function is represented, the worse the corresponding scene advantages and disadvantages are, the better the automatic driving function is represented, and the better the corresponding scene advantages and disadvantages are.
after the scene quality is defined, the initial high-efficiency clustering test scene analysis and determination are carried out, and the method comprises the following steps:
s5-1: dividing a test process of the to-be-tested automatic driving vehicle under the field into a plurality of interaction segments according to an evolution process, analyzing other road participants having a mutual relation with the to-be-tested automatic driving vehicle under each interaction segment aiming at each interaction segment, and determining the maximum number of the road participants;
S5-2: determining a clustering region where each other road participant can be located, and combining scene parameterization to merge and simplify the clustering regions;
S5-3: and determining an initial high-efficiency clustering test scene according to the clustering areas where all other road participants can be located and by combining the actual test field and the test conditions.
The initial high-efficiency clustering test scene refers to determination of the number of road participants (including to-be-tested automatic driving vehicles) in the scene, determination of initial relative positions of the road participants (including to-be-tested automatic driving vehicles), but the specific starting position of each road participant and behaviors of each road participant in the test process are uncertain. In the test, the test coverage of the automatic driving function to be tested is required to be ensured to be as high as possible, and the test coverage of the function to be tested is formed by combining the initial high-efficiency clustering test scene with the specific behaviors of road participants except the automatic driving vehicle to be tested. The main target of the initial high-efficiency clustering test scene selection is to determine the initial positions of other road participants, in order to obtain an initial high-efficiency clustering test scene, the test process under the automatic driving vehicle field is divided into a plurality of interaction segments according to the evolution process, the road participants which have mutual relations with the automatic driving vehicle under each interaction segment are analyzed aiming at each interaction segment, the maximum number of the road participants is determined, then the clustering regions in which the road participants can be located are determined, the road participants in the clustering regions only in one clustering region can be represented by a set of continuous parameters, so that the search of the subsequent poor scenes is facilitated, and finally, one or a group of initial high-efficiency clustering test scenes is determined by combining the conditions of the actual field and the test equipment.
further, if a group of initial high-efficiency clustering test scenes simultaneously appear, one of the initial high-efficiency clustering test scenes is further selected as a final initial high-efficiency clustering test scene, the selection principle is based on the same analogy condition, the complexity of each initial high-efficiency clustering test scene on the automatic driving function to be tested is transversely compared, and the most complex scene is selected as the final initial high-efficiency clustering test scene.
the automatic driving function test is based on the quick search of the underground poor scene of a test field, adopts a mode of combining the online road participant behavior control and the offline road participant behavior strategy optimization, gradually approaches the worst scene of an automatic driving vehicle through a field test, and comprises the following steps:
S6-1: setting test times and scene goodness threshold values; determining interference decision parameters and initial parameters of other road participants except the to-be-tested automatic driving vehicle and site initial parameters of the to-be-tested automatic driving vehicle;
s6-2: determining behavior decisions of other road participants according to the interference decision parameters, uploading the behavior decisions to a control system through a perception system, and controlling behaviors of the other road participants in a test scene;
S6-3: completing a first test of the to-be-tested automatic driving vehicle and other road participants in an initial high-efficiency clustering test scene, and recording test data;
s6-4: calculating scene quality aiming at the data of the step S6-3;
S6-5: if the scene quality of the initial field test is worse than the threshold value, the test scene is a worse scene, the performance of the automatic driving function to be tested in the scene does not meet the test requirement, and the test is finished;
if the primary site test scene quality is better than the threshold value, the test scene is not a poor scene, the countermeasure enhancement module is utilized to adjust the interference decision parameters of other road participants and the initial behavior parameters of each road participant, the goal is to make the scene quality worse, the step S6-2 to the step S6-4 are returned, the test scene quality is calculated again, the process is repeated, the scene quality is continuously worsened, and further analysis is carried out:
1) if the scene quality is worse than the threshold before the test times reach the preset value, the test scene is a worse scene aiming at the automatic driving function to be tested, and the automatic driving function to be tested is not perfect;
2) If the scene quality is not worse than the threshold value when the test times reach the preset value, it is indicated that a poor scene proving that the automatic driving function to be tested is incomplete is not found, and the automatic driving function to be tested is relatively complete.
Compared with the prior art, the method has the remarkable beneficial effects that the automatic driving function to be tested is tested by applying an actual test field, so that the test conclusion is closer to the real situation; and the automatic driving function test is carried out on the basis of continuously searching poor test scenes, and improvement suggestions are directly provided for the perfection degree of the automatic driving function.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is an automatic lane change test scenario;
FIG. 2 is a representation of the maximum number of interfering vehicles initially associated with lane change;
FIG. 3 is a representation of the maximum number of associated interfering vehicles in the middle of a lane change;
FIG. 4 is a lane change interfering vehicle zone representation;
FIG. 5 is an initial high-efficiency clustering test scenario of 8 kinds of lane change interference;
FIG. 6 is a final determined channel change interference initial high-efficiency clustering test scenario;
Fig. 7 is an automatic driving function test evaluation flow based on an initial efficient cluster test scenario.
Detailed Description
the invention is described in detail below with reference to the drawings, which form a part hereof, and which are shown by way of illustration, embodiments of the invention. However, it should be understood by those skilled in the art that the following examples are not intended to limit the scope of the present invention, and any equivalent changes or modifications made within the spirit of the present invention should be considered as falling within the scope of the present invention.
The following describes an embodiment of the present invention in detail by performing an in-field test evaluation analysis on the lane change function of an autonomous vehicle.
Determination of automatic driving function to be tested and site test conditions
The method comprises two contents of analysis of an application scene of the automatic driving function to be tested and determination of a test field and an auxiliary test facility:
Firstly, analyzing an application scene of a lane changing function of an automatic driving vehicle to be detected, for example, assuming that the lane changing condition of the automatic driving vehicle to be detected is as follows: the speed of the vehicle is 0-100km/h, the speed of other vehicles in the road is 0-100km/h, other road participants except motor vehicles do not exist in the road, the road type is a straight road, the road width is 3.5-3.75 m, the weather condition is clear weather, and the application scene is determined.
Secondly, determining test conditions which can be provided by a test site, for example, determining road traffic environment and auxiliary test facilities such as a soft target guide vehicle which can be provided by the test site, assuming that the test site can provide a bidirectional 6-lane straight road with the road width of 3.75 and the length of 800m, and other road participants are 4 soft target guide vehicles which can be used as interference vehicles to participate in the test, so that the maximum number of the interference vehicles which can be provided in a site test scene is 4.
Second, test scenario parameterization
and (3) carrying out site test scene parameterization on a corresponding test scene aiming at the automatic driving function and the test site to be tested, and determining static scene parameters, dynamic road participants and other related parameters in the scene, so as to define the scene quality and design an initial high-efficiency cluster test scene.
For the above automatic lane changing function to be tested and the site conditions, the definition of the scene goodness and badness and the design of the initial high-efficiency clustering test scene are considered at the same time, and the following parameterization can be performed on the test scene, as shown in fig. 1.
in fig. 1, for the autonomous vehicle VUT to be tested, the parameters to be determined include its initial position (X 0, Y 0) when receiving a lane change command, the initial vehicle speed v 0 and the target lane, and the later track of the autonomous vehicle to be tested is generated by the autonomous vehicle itself according to a specific scene, i.e. so-called autonomous driving, which is not considered here, for other road participants, such as an interfering vehicle, the number n of interfering vehicles needs to be determined, and the track of each interfering vehicle (X 11, Y 11) - (X ij, Y ij), i represents the own lane and all the interfering lanes, generally i is 1-3, i represents the own lane and two lanes on the left and right, and j represents the jth interfering vehicle on a certain lane.
third, defining the quality of specific test scene based on automatic driving function performance
the method comprises the steps of defining the quality of a specific test scene on the basis of scene parameterization, and quantifying the quality of an automatic driving function in a site test through the quality of the scene, wherein in a lane change test, for example, when the function which is most concerned is safety, the quality of the scene can be defined as the risk of the scene. The definition of the scene quality includes how to deduce the quantitative scene quality through data calculation of a specific primary test, so that comparison of different test scenes can be performed, and the worse the automatic driving function is, the worse the corresponding scene quality is, the better the automatic driving function is, the better the corresponding scene quality is.
the method for solving the global risk S dan through the site test data is divided into the following three steps:
(1) and calculating the vehicle-vehicle global risk degree S dan-vm, wherein the maximum vehicle-vehicle global risk degree MS dan-vm is determined by the maximum value of the collision risk degree of the to-be-detected automatic driving vehicle and each interference vehicle at each moment, and the expression is as follows:
MSdan-vm＝max{sup(Sdan-1(t)),sup(Sdan-2(t)),...,sup(Sdan-i(t))} (1)
wherein S dan-i (t) is the risk of the automatic driving vehicle to be tested and the ith interference vehicle at the time t, sup is used for collecting the upper bound value of S dan-i (t), and the expression of S dan-i (t) is as follows:
in the formula, (x v (t), y v (t)), V v (t) is a position coordinate and a vehicle speed of a vehicle to be measured at the time t, (x i (t), y i (t)), and V i (t) are a position coordinate and a vehicle speed of an i-th vehicle in an interfering vehicle at the time t, W is a vehicle width, W L is a road width, and C V is a constant coefficient, the purpose of setting the coefficient is to ensure that (| x v (t) -x i (t) | + (V i (t) -V v (t)) + C V) and (| x v (t) -x i (t) | - (V i (t) -V v (t)) + C V) are not negative, and C V takes a sufficiently large value;
(2) And calculating a lane departure global risk S dan-lm, wherein the maximum lane departure global risk MS dan-lm is determined by the maximum departure risk of the vehicle to be detected relative to the lane boundary at each moment, and is expressed as follows:
MSdan-lm＝max{sup(Sdan-llane(t)),sup(Sdan-rlane(t))} (4)
In the formula, S dan-llane (t) and S dan-rlane (t) are lane departure risk degrees of the vehicle to be tested at the time of the left and right boundaries t of any relevant lane g, respectively, sup also represents a corresponding upper boundary value, and the expressions of S dan-llane (t) and S dan-rlane (t) are as follows:
In the formula, y g is a y coordinate of the left or right boundary of the corresponding lane g when calculating S dan-llane (t) and S dan-rlane (t), C l is a constant coefficient set to ensure that (— | x v (t) -x i (t) | + V v (t) + C l) is not negative, C l is a sufficiently large value, and other parameters are as above;
(3) Calculating a global risk S dan, wherein the worst global risk MS dan is expressed as follows:
MSdan＝max{MSdan-vm,MSdan-lm} (6)。
Analyzing and designing initial high-efficiency clustering test scene
after the scene quality is defined, the initial high-efficiency clustering test scene analysis and design are required. The initial high-efficiency clustering test scene refers to determination of the number of road participants (including to-be-tested automatic driving vehicles) and determination of initial relative positions of the road participants (including to-be-tested automatic driving vehicles) in the scene, but specific starting positions of the road participants and behaviors of the road participants in the test process are uncertain. The test coverage of the to-be-tested automatic driving function is required to be ensured to be as high as possible in the test, so that the test coverage of the to-be-tested function is formed as high as possible by combining the initial high-efficiency clustering test scene with the specific behaviors of road participants except for the to-be-tested automatic driving vehicle.
the main target of the initial high-efficiency clustering test scene selection is to determine the initial positions of other road participants, in order to obtain the initial high-efficiency clustering test scene, firstly, the test process under the automatic driving vehicle field needs to be divided into a plurality of interaction segments according to the evolution process, the road participants having mutual relations with the automatic driving vehicle under each interaction segment are analyzed aiming at each interaction segment, the maximum number of the road participants is determined, then the clustering regions where the road participants can be located are determined, the road participants in one clustering region can be represented by a set of continuous parameters, so that the search of the subsequent poor scene is facilitated, and finally, one or a group of initial high-efficiency clustering test scenes is determined by combining the actual field and the conditions of the test equipment.
after the initial high-efficiency clustering test scenes are determined, if a group of initial high-efficiency clustering test scenes appear, one of the initial high-efficiency clustering test scenes needs to be further selected as subsequent poor scene search. And for a group of initial high-efficiency clustering test scenes, transversely comparing the complexity of each initial logic test scene to the automatic driving function to be tested, so that the complexity evaluation of each initial high-efficiency clustering test scene has certain comparability, and the basic initial conditions of each scene need to be similar.
Taking the lane changing working condition as an example, in order to ensure the test coverage degree of the function as high as possible in the test, the coverage degree of the test scene as high as possible needs to be formed by combining the initial high-efficiency clustering test scene with the track (including the speed) of the interfering vehicle, the initial position of the interfering vehicle is the main target selected by the initial high-efficiency clustering test scene, and the initial position can be obtained by analyzing each stage of the lane changing process.
First, the autonomous vehicle itself can be in the high, medium and low speed lanes at the time of lane change, but the test for this function can be generalized to two cases of high speed change and low speed change.
Secondly, considering a clustering region in the scene, in which the interfering vehicles may affect the automatic driving vehicle in the lane changing process, analyzing the effective interfering vehicles with the largest number of lanes according to the evolution of the automatic driving vehicle in the lane changing process, thereby obtaining a reasonable number of initial high-efficiency clustering test scenes. Taking the automatic driving vehicle to change the high speed and the low speed as an example, the automatic driving lane changing process is divided into three processes of lane changing starting, lane changing process and lane changing ending. In the initial stage of lane change, interfering vehicles which have a mutual relationship with the automatically-driven vehicles appear in 5 cluster regions at most, such as regions a to E shown in fig. 2, wherein a denotes the rear of the original lane of the automatically-driven vehicles, B denotes the front of the original lane, D denotes a lane change region which has an interference with the transverse position of the automatically-driven vehicles, and C and E are regions in front of and behind D in the lane change region, respectively.
Fig. 3 shows the areas with the most interfering vehicles in the middle lane-changing period, which are six areas a to F, and are respectively 6 area positions of the left front, the left rear, the right front, the right rear, the right left and the right of the position of the automatic driving vehicle.
when the lane change process is over, whether the lane change is successful or failed, one test has ended, so the maximum number of relevant interfering vehicles is no longer considered. The method integrates two conditions of initial lane changing and middle lane changing, and in the automatic lane changing high-speed and low-speed process, the maximum number of the interference vehicles related to the high-speed lane is 3, and the maximum number of the interference vehicles related to the low-speed lane is 3.
it can be seen that the initial regions of interconnected vehicles are unified into a region where the specific initial position of the interfering vehicle is a continuous variable, so the selection of the number of interfering vehicles for the initial high-efficiency cluster test scenario can be changed to the A, B and C region determination shown in fig. 4, which is basically the vehicle in front of and behind the autonomous vehicle in the original lane, the vehicle in front of and behind the autonomous vehicle in the lane change lane.
According to the analysis of the maximum number of the relevant interfering vehicles, the sum of the numbers of vehicles in the A, B clustering areas is 3 (as shown in fig. 3), and the maximum number of the C clustering areas is also 3. When the number of the interference vehicles in one clustering area is determined, the condition that the number of the interference vehicles in one clustering area is large can cover the condition that the number is small, and firstly, when the initial position of one interference vehicle is far and the subsequent position is continuously far away from the automatic driving vehicle, the interference vehicle can be considered not to influence the automatic driving vehicle to be detected; and secondly, when the number of the interference vehicles in the clustering areas A and B is larger than 2, the effect on the automatic driving vehicle, which can be generated by more than 2 interference vehicles, can be covered by two interference vehicles which are close to the automatic driving vehicle.
therefore, assuming that the number of interfering vehicles in each cluster area is n A, n B and n C, since n A, n B and n C are integers, the number of combinations satisfying the following conditions is limited, so the determination of the positions of the interfering vehicles can be simplified to solve the number of combinations satisfying the following conditions:
nA≤2；nB≤2
nA+nB≤3
nC≤3
nA+nB+nC＝4
The reason why n A + n B + n C is 4 is that the maximum number of the interfering vehicles which can be provided by the field energy is 4 when the scene is designed, so that the scene is as complex as possible, and 4 interfering vehicles are added into the test, so 8 initial high-efficiency cluster test scenes are finally obtained, as shown in fig. 5.
After a group of initial high-efficiency cluster test scenes is determined, the initial high-efficiency cluster test scenes which have complex performance on the to-be-tested automatic driving vehicle need to be further selected. In order to enable the complexity evaluation of each initial high-efficiency clustering test scene to have certain comparability, basic initial conditions of each scene need to be set to be similar, the initial speed of each interference vehicle is selected, the distances between the interference vehicles and the automatic driving vehicles to be tested are all consistent, each interference vehicle is set to keep the speed unchanged, the test is completed, the danger values of the test scenes are compared, the scene with the highest danger is selected, and finally the initial high-efficiency clustering test scene shown in fig. 6 is selected as the test scene.
Fifthly, fast searching and automatic driving function testing evaluation of scenes with poor automatic driving function
And carrying out rapid search under the site test of the scene with poor automatic driving function, wherein the adopted method is a mode of combining online road participant behavior control and offline road participant behavior strategy optimization, and gradually approaches the worst scene of the automatic driving vehicle through the site test.
The automatic driving function test evaluation is carried out under the condition of limited test times and limited poor scene search, and the test times and the preset value of the test scene quality are set before the test.
after the initial high-efficiency clustering test scene is determined, a group of interference behavior decision parameters and initial parameters of road participants except the automatic driving vehicle to be tested and initial parameters of the automatic driving vehicle to be tested are respectively determined. And different automatic driving functions to be tested, and the decision parameters of the interference behaviors of the road participants are different from the initial behavior parameters. And acquiring behavior decision input through a perception system, determining the behavior decision of the road participant according to the parameters, transmitting the behavior decision to a control layer, and finally controlling the behavior of the corresponding road participant in a test scene. The perception information can be collected by the equipment according to the test site condition and the test equipment condition, and can also be collected by the site equipment and then transmitted to the road participants, or comes from information interaction among the road participants. The behavior of the autonomous vehicle to be tested is influenced by the autonomous driving function and the initial parameters to be tested. And combining other scene factors, such as road conditions, weather conditions, lighting conditions and the like, to form a determined field test. And obtaining field test data after the field test, wherein the test data comprises the track, speed, acceleration and other information of the to-be-tested automatic driving vehicle and other road participants, and scene quality parameters expressed for the to-be-tested function in the field test.
If the scene quality parameter expressed by the function to be tested in the test is better than the preset value, it indicates that the performance of the automatic driving vehicle in the test scene determined this time is not enough to indicate the deficiency of the function to be tested, and at the same time, it also indicates that the test scene is not a poor scene, at this time, the test data needs to be fed back to the confrontation enhancement module set by the computer control system, and the confrontation enhancement module adjusts the interference decision parameter of other road participants and the initial behavior parameter of each road participant according to the behavior feedback of the automatic driving vehicle to other road participants, and expects to develop toward the direction of poor quality of the test scene in the next test, thereby gradually approaching the poor situation that the automatic driving function may appear in the application scene.
With each parameter adjustment, if the quality parameter of the test scene expressed by the function to be tested in the test is worse than the preset value before reaching the preset value of the test times, the worse test scene is found, and the automatic driving function is not perfect, the automatic driving function to be tested can be analyzed by combining the obtained scene with the worst quality, an improvement suggestion is provided, the purpose of the test is achieved, and the test is completed.
if the scene quality parameter expressed by the to-be-tested function is not different from the preset value all the time when the test times reach the preset value, the to-be-tested automatic driving function can operate under reasonable performance indexes in most scenes, and the poor test scene is not searched all the time.
for example, for the embodiment of the automatic driving lane changing function, a mode of combining online interference vehicle trajectory planning and offline interference vehicle trajectory planning strategy optimization is adopted, and a worst scenario of an automatic driving vehicle is gradually approached through an actual field test, and a specific implementation flow is shown in fig. 7.
the number of trials and the worst global risk threshold are set first. Respectively determining a group of interference vehicle track interference decision parameters and initial parameters of the to-be-tested automatic driving vehicle on the basis of determining the initial high-efficiency clustering test scene; an interference vehicle track decision determined by the track interference decision parameter is used for obtaining required decision information through a sensing layer and transmitting the decision to a control layer, and finally the track of the interference vehicle in a test scene is controlled; and forming a determined site test by combining other scene factors. And obtaining field test data through vehicle-mounted or field measuring equipment after the field test, wherein the test data comprises information such as the track, the speed, the acceleration and the like of the automatic driving vehicle to be tested and the interference vehicle, and calculating the worst global risk value.
If the worst global risk degree of the initial site test is worse than the threshold value, the test scene is a worse scene, the performance of the automatic driving lane changing function to be tested in the scene does not meet the test requirement, and the test is finished;
If the worst global risk of the initial site test is better than the threshold, it indicates that the risk presented by the automatic driving vehicle in the test scene determined this time is not enough to indicate that a lane change may occur, and also indicates that the test scene is not a poor test scene, at this time, test data needs to be fed back to a countermeasure enhancement module (a calculation module arranged in a computer), and the countermeasure enhancement module adjusts an interference decision parameter and an initial parameter of the interference vehicle according to the behavior feedback of the automatic driving vehicle to the interference vehicle, so as to gradually search for a scene where the worst global risk is worse than the threshold, thereby gradually approaching the risk that the lane change function may occur in an application scene.
With each parameter adjustment, 1) if the worst global risk difference is larger than a threshold value before the test times reach a preset value, the test scene at the moment is a poorer test scene, and the automatic driving lane changing function to be tested is not complete; 2) if the worst global risk degree is not worse than the threshold value when the test times reach the preset value, it is indicated that a poor scene proving that the automatic driving function to be detected is incomplete is not found, and the automatic driving lane changing function to be detected is relatively complete.
Claims (4)
1. An automatic driving function test evaluation method based on poor scene search under a field is characterized by comprising the following steps:
S1: analyzing an application scene required by the performance of the automatic driving function to be tested;
S2: determining a test site condition based on application scene analysis;
S3: parameterization of a test scene under the determined test site condition comprises static scene parameterization and dynamic road participant parameterization;
s4: defining scene quality based on the automatic driving function performance to be tested and the test scene parameterization;
s5: analyzing and determining an initial high-efficiency clustering test scene based on the automatic driving function performance to be tested and the parameterization of the test scene, wherein the analysis and determination comprise the determination of the number of road participants and the determination of the initial relative position of the road participants;
s6: and the automatic driving function test based on poor scene search under the scene quality definition and the initial high-efficiency clustering test scene.
2. The method of claim 1, wherein the initial high-efficiency cluster test scenario analysis and determination comprises the steps of:
s5-1: dividing a test process of the to-be-tested automatic driving vehicle under the field into a plurality of interaction segments according to an evolution process, analyzing other road participants having a mutual relation with the to-be-tested automatic driving vehicle under each interaction segment aiming at each interaction segment, and determining the maximum number of the road participants;
s5-2: determining a clustering region where each other road participant can be located, and combining scene parameterization to merge and simplify the clustering regions;
S5-3: and determining an initial high-efficiency clustering test scene according to the clustering areas where all other road participants can be located and by combining the actual test field and the test conditions.
3. the method of claim 2, wherein if a set of initial high-efficiency clustering test scenarios simultaneously appear, one of the initial high-efficiency clustering test scenarios is further selected as a final initial high-efficiency clustering test scenario, the selection principle is based on the same analogy condition, the complexity of each initial high-efficiency clustering test scenario for the automatic driving function to be tested is transversely compared, and a most complex scenario is selected as the final initial high-efficiency clustering test scenario.
4. the method of claim 1, wherein the automated driving function test based on the definition of scene goodness and inferiority and poor scene search in the initial efficient cluster test scene comprises the following steps:
S6-1: setting test times and scene goodness threshold values;
Determining interference decision parameters and initial parameters of other road participants except the to-be-tested automatic driving vehicle and in-field initial parameters of the to-be-tested automatic driving vehicle;
s6-2: determining behavior decisions of other road participants according to the interference decision parameters, uploading the behavior decisions to a control system through a perception system, and controlling behaviors of the other road participants in a test scene;
s6-3: completing a first test of the to-be-tested automatic driving vehicle and other road participants in an initial high-efficiency clustering test scene, and recording test data;
S6-4: calculating scene quality aiming at the data of the step S6-3;
S6-5: if the scene quality of the initial field test is worse than the threshold value, the test scene is a worse scene, the performance of the automatic driving function to be tested in the scene does not meet the test requirement, and the test is finished;
if the primary site test scene quality is better than the threshold value, the test scene is not a poor scene, the countermeasure enhancement module is utilized to adjust the interference decision parameters of other road participants and the initial behavior parameters of each road participant, the goal is to make the scene quality worse, the step S6-2 to the step S6-4 are returned, the test scene quality is calculated again, the process is repeated, the scene quality is continuously worsened, and further analysis is carried out:
1) if the scene quality is worse than the threshold before the test times reach the preset value, the test scene is a worse scene aiming at the automatic driving function to be tested, and the automatic driving function to be tested is not perfect;
2) If the scene quality is not worse than the threshold value when the test times reach the preset value, it is indicated that a poor scene proving that the automatic driving function to be tested is incomplete is not found, and the automatic driving function to be tested is relatively complete.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910720260.9A CN110553853B (en) | 2019-08-06 | 2019-08-06 | Automatic driving function test and evaluation method based on poor scene search under field |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910720260.9A CN110553853B (en) | 2019-08-06 | 2019-08-06 | Automatic driving function test and evaluation method based on poor scene search under field |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110553853A true CN110553853A (en) | 2019-12-10 |
| CN110553853B CN110553853B (en) | 2020-11-20 |
Family
ID=68736959
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910720260.9A Active CN110553853B (en) | 2019-08-06 | 2019-08-06 | Automatic driving function test and evaluation method based on poor scene search under field |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110553853B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111144015A (en) * | 2019-12-30 | 2020-05-12 | 吉林大学 | Method for constructing virtual scene library of automatic driving automobile |
| CN111444605A (en) * | 2020-03-24 | 2020-07-24 | 上海汽车集团股份有限公司 | Method and device for calculating coverage of danger test scene |
| CN111915888A (en) * | 2020-07-14 | 2020-11-10 | 同济大学 | Method for calculating complexity of traffic participants in automatic driving test scene |
| JP2021107782A (en) * | 2019-12-27 | 2021-07-29 | 日産自動車株式会社 | Design support method and design support device |
| CN113589798A (en) * | 2021-08-12 | 2021-11-02 | 上海裹动科技有限公司 | Automatic test behavior generation method and server |
| CN114838948A (en) * | 2022-03-04 | 2022-08-02 | 湖北国际物流机场有限公司 | Automatic lane merging test method and system for airport automatic driving guide vehicle |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE4124654A1 (en) * | 1991-07-25 | 1993-01-28 | Bundesrep Deutschland | Continuous automatic vehicle orientation on road - using monocular image and modelling to estimate road curvature and width from geometry and dynamic aspects of scene |
| CN107727411A (en) * | 2017-10-30 | 2018-02-23 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle test and appraisal scene generation system and method |
| CN107782564A (en) * | 2017-10-30 | 2018-03-09 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle evaluation system and method |
| CN107843440A (en) * | 2017-10-30 | 2018-03-27 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle Performance Test System and method |
| CN108765235A (en) * | 2018-05-09 | 2018-11-06 | 公安部交通管理科学研究所 | Automatic driving vehicle test scene construction method and test method based on the destructing of traffic accident case |
| US20190086926A1 (en) * | 2017-09-18 | 2019-03-21 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for exporting driving property index of autonomous vehicle |
| CN109753623A (en) * | 2018-12-10 | 2019-05-14 | 清华大学 | A kind of analysis of automatic driving vehicle multi-test scene and number simplifying method |
| CN109948289A (en) * | 2019-04-01 | 2019-06-28 | 清华大学 | The vehicle autonomous parking Function Appraising method of rate search is occupied based on the maximum overall situation |
| WO2019132930A1 (en) * | 2017-12-28 | 2019-07-04 | Intel Corporation | System and method for simulation of autonomous vehicles |
| CN109992884A (en) * | 2019-04-01 | 2019-07-09 | 清华大学 | Automatic driving vehicle Function Appraising method based on worst scene search |
| CN110020797A (en) * | 2019-03-27 | 2019-07-16 | 清华大学苏州汽车研究院（吴江） | The evaluation method of automatic Pilot test scene based on perception defect |
| US20190271614A1 (en) * | 2018-03-01 | 2019-09-05 | RightHook, Inc. | High-Value Test Generation For Autonomous Vehicles |
-
2019
- 2019-08-06 CN CN201910720260.9A patent/CN110553853B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE4124654A1 (en) * | 1991-07-25 | 1993-01-28 | Bundesrep Deutschland | Continuous automatic vehicle orientation on road - using monocular image and modelling to estimate road curvature and width from geometry and dynamic aspects of scene |
| US20190086926A1 (en) * | 2017-09-18 | 2019-03-21 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for exporting driving property index of autonomous vehicle |
| CN107727411A (en) * | 2017-10-30 | 2018-02-23 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle test and appraisal scene generation system and method |
| CN107782564A (en) * | 2017-10-30 | 2018-03-09 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle evaluation system and method |
| CN107843440A (en) * | 2017-10-30 | 2018-03-27 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle Performance Test System and method |
| WO2019132930A1 (en) * | 2017-12-28 | 2019-07-04 | Intel Corporation | System and method for simulation of autonomous vehicles |
| US20190271614A1 (en) * | 2018-03-01 | 2019-09-05 | RightHook, Inc. | High-Value Test Generation For Autonomous Vehicles |
| CN108765235A (en) * | 2018-05-09 | 2018-11-06 | 公安部交通管理科学研究所 | Automatic driving vehicle test scene construction method and test method based on the destructing of traffic accident case |
| CN109753623A (en) * | 2018-12-10 | 2019-05-14 | 清华大学 | A kind of analysis of automatic driving vehicle multi-test scene and number simplifying method |
| CN110020797A (en) * | 2019-03-27 | 2019-07-16 | 清华大学苏州汽车研究院（吴江） | The evaluation method of automatic Pilot test scene based on perception defect |
| CN109948289A (en) * | 2019-04-01 | 2019-06-28 | 清华大学 | The vehicle autonomous parking Function Appraising method of rate search is occupied based on the maximum overall situation |
| CN109992884A (en) * | 2019-04-01 | 2019-07-09 | 清华大学 | Automatic driving vehicle Function Appraising method based on worst scene search |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2021107782A (en) * | 2019-12-27 | 2021-07-29 | 日産自動車株式会社 | Design support method and design support device |
| JP7395346B2 (en) | 2019-12-27 | 2023-12-11 | 日産自動車株式会社 | Design support method and design support device |
| CN111144015A (en) * | 2019-12-30 | 2020-05-12 | 吉林大学 | Method for constructing virtual scene library of automatic driving automobile |
| CN111444605A (en) * | 2020-03-24 | 2020-07-24 | 上海汽车集团股份有限公司 | Method and device for calculating coverage of danger test scene |
| CN111915888A (en) * | 2020-07-14 | 2020-11-10 | 同济大学 | Method for calculating complexity of traffic participants in automatic driving test scene |
| CN113589798A (en) * | 2021-08-12 | 2021-11-02 | 上海裹动科技有限公司 | Automatic test behavior generation method and server |
| CN114838948A (en) * | 2022-03-04 | 2022-08-02 | 湖北国际物流机场有限公司 | Automatic lane merging test method and system for airport automatic driving guide vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110553853B (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110553853B (en) | Automatic driving function test and evaluation method based on poor scene search under field | |
| CN110834644B (en) | Vehicle control method and device, vehicle to be controlled and storage medium | |
| CN109164785B (en) | Emulation test method, device, equipment and the storage medium of automatic driving vehicle | |
| CN107168303A (en) | A kind of automatic Pilot method and device of automobile | |
| DE102016113903A1 (en) | Vehicle route determination | |
| CN109753623B (en) | Method for analyzing multiple test scenes and simplifying number of automatic driving vehicles | |
| CN113665574B (en) | Intelligent automobile lane change duration prediction and anthropomorphic track planning method | |
| CN109992884B (en) | Automatic driving vehicle function evaluation method based on worst scene search | |
| CN112829747A (en) | Driving behavior decision method and device and storage medium | |
| CN115735107A (en) | Computer-implemented method for providing a test procedure for a traffic scenario to be tested | |
| CN113665593A (en) | Longitudinal control method and system for intelligent driving of vehicle and storage medium | |
| US11255687B2 (en) | Method for trajectory planning of a movable object | |
| CN114475608A (en) | Method and device for changing lanes for automatic driving vehicle, vehicle and storage medium | |
| CN114492043A (en) | Personalized driver following modeling method considering perception limited characteristics | |
| CN111123729A (en) | Method and device for vehicle driving simulation optimization, storage medium and terminal equipment | |
| Son et al. | A simulation-based testing and validation framework for ADAS development | |
| Kaur et al. | Scenario-based simulation of intelligent driving functions using neural networks | |
| CN114475607B (en) | Method and device for changing lanes for automatic driving vehicle, vehicle and storage medium | |
| CN115923833A (en) | Personifying decision control method and device for vehicle, vehicle and storage medium | |
| CN116107252A (en) | Calibration method and device for vehicle autonomous lane change parameters, electronic equipment and medium | |
| US20220223047A1 (en) | Computer-implemented method for determining similarity values of traffic scenarios | |
| CN115451987A (en) | Path planning learning method for automatic driving automobile | |
| CN115439816A (en) | Unmanned system steering angle safety verification method based on convex optimization | |
| EP3920070A1 (en) | Testing and simulation in autonomous driving | |
| CN110008577B (en) | Vehicle automatic lane changing function evaluation method based on worst global risk degree search |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |