CN111841012B - Automatic driving simulation system and test resource library construction method thereof - Google Patents
Automatic driving simulation system and test resource library construction method thereof Download PDFInfo
- Publication number
- CN111841012B CN111841012B CN202010581488.7A CN202010581488A CN111841012B CN 111841012 B CN111841012 B CN 111841012B CN 202010581488 A CN202010581488 A CN 202010581488A CN 111841012 B CN111841012 B CN 111841012B
- Authority
- CN
- China
- Prior art keywords
- test
- scene
- traffic
- driving
- simulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 145
- 238000004088 simulation Methods 0.000 title claims abstract description 88
- 238000010276 construction Methods 0.000 title abstract description 12
- 238000011156 evaluation Methods 0.000 claims abstract description 40
- 230000003068 static effect Effects 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 238000012216 screening Methods 0.000 claims description 16
- 230000003993 interaction Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 6
- 230000008447 perception Effects 0.000 claims description 5
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 206010039203 Road traffic accident Diseases 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 11
- 230000006399 behavior Effects 0.000 description 10
- 238000011160 research Methods 0.000 description 9
- 238000011161 development Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 6
- 238000013461 design Methods 0.000 description 4
- 238000013210 evaluation model Methods 0.000 description 4
- 238000005094 computer simulation Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000010355 oscillation Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000004836 empirical method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/80—Special adaptations for executing a specific game genre or game mode
- A63F13/803—Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Traffic Control Systems (AREA)
Abstract
An automatic driving simulation system and a method for constructing a test resource library thereof. The automatic driving simulation system is used for constructing a test environment based on a game engine such as a fantasy 4 engine, creatively provides a dynamic and static combined simulation test resource library construction method, realizes high-reality dynamic scene simulation, and finally combines the experience of domestic and foreign automatic driving evaluation and the L1-L3 level automatic driving test characteristics to construct an automatic driving system performance comprehensive evaluation system. The invention provides a dynamic and static combined simulation test resource library construction method.
Description
Technical Field
The invention relates to the field of traffic, in particular to an analog simulation system for an automatic driving vehicle.
Background
The automatic driving technology is used as a systematic engineering, the related modules are numerous, the business logic is complex, and the real vehicle on-road test is required to ensure the overall safety. However, this is also a major challenge in the development of current autopilot systems-real vehicle drive tests "road resistance and length". According to the data in the Chinese automatic driving simulation technology research report (2019): a fleet equipped with a 1000 autopilot test vehicle took about 50 years to complete a sufficient mileage test. Enterprises are difficult to continuously input huge resources to support wide real-vehicle road testing, and in practice, wide road testing is difficult to cover complex and diverse traffic scenes, and meanwhile, frequent real-vehicle road testing brings huge potential safety risks for traffic safety. For example, in 2016, the Uber vehicle has been crashing into a pedestrian that suddenly appears in the center of the road. Therefore, the automatic driving virtual simulation system is paid attention to by various research institutions for early verification algorithms, and possible technical problems are solved before the real vehicle drive test, so that the development of the automatic driving system is accelerated.
The automatic driving simulation test is to build a mathematical model of a real static and dynamic traffic scene by a computer simulation technology, so that an automatic driving automobile and an algorithm perform driving test in a virtual traffic scene. Compared with the real lane road test, the simulation test is easier, the test cost can be effectively reduced, and the algorithm development iteration period is shortened; meanwhile, a traffic test scene with strong customization pertinence can be constructed through simulation software, so that algorithm specificity upgrading iteration and reinforcement are realized; meanwhile, for some scenes with high reproduction cost, such as highways under rainy and snowy weather, a physical test scene is not required to be built or test occasions such as rainy and snowy weather are specially waited, simulation is greatly reduced in implementation difficulty of the test scene, and the test breadth is enlarged. At present, the unmanned high-efficiency test flow is that an algorithm is upgraded on computer software through a simulation test technology, so that the unmanned high-efficiency test flow can run successfully in a virtual environment, then a closed field test is carried out, and finally an open road test is carried out.
At present, an automatic driving simulation technology is being introduced into development processes by more and more unmanned vehicle research enterprises and scientific research institutions, for example, a Waymo unmanned vehicle takes a simulation test as a necessary project before carrying out a public road test; the hundred degrees list the simulation platform as an Apollo main variation way, and create income by providing simulation cloud service for Apollo partners; a number of automated driving beginners such as roadstar ai, pony ai, etc. are on their own to develop simulators. The Chinese autopilot simulation technology research report (2019) indicates that the total scale of simulation software and the international market for testing in the next 5 years is about one billion dollars, and the commercial simulation software meeting the autopilot algorithm development and the whole vehicle system integration is one of the bottom basic tools on the research and development chain. China also lays a strategy for developing the automatic driving simulation technology. In the early 2020, the "intelligent automobile innovation development strategy" issued by 11 Committee of the industry and informatization department and the like clearly requires to develop technology and verification tools such as virtual simulation, software and hardware combined simulation, real lane testing and the like, and a multi-level test evaluation system.
The premise of the high-reliability automatic driving simulation test is a reasonable and diversified test scene, and the simulation test can fully mine the performance potential of the automatic driving system and expose algorithm loopholes at the moment. The existing domestic and foreign simulation test scene design is basically concentrated on reproducing important natural driving scenes, and the method can test the safety of an automatic driving system to a certain extent, but ignores other dangerous boundary situations. More importantly, the current test scene design cannot perfectly solve the simulation of dynamic elements in the scene, such as motor vehicle simulation, pedestrian and non-motor vehicle simulation, which involve other traffic participants. The existing dynamic simulation scene simulation systems such as a simulation world Carcraft of Waymo, an open source simulation test environment CARLA and the like all have the problems that simulation traffic participants lack interaction with a tested vehicle in a simulation test, and a generalization method of a real scene lacks deep research and the like. In the simulation test process, the vehicle in the simulation environment runs according to the actual measurement or simulation track, and the lack of interaction with the tested vehicle or the generated interaction is unreasonable and unrealistic. However, the problem of loss of reality exists in the generalization of the simulation dynamic scene to the real scene, for example, after a dense traffic flow case is generalized and a vehicle track is changed, a plurality of tracks around the vehicle track are actually affected and spread, the driving disturbance of a bicycle sometimes causes instability of the whole traffic flow, and a simple generalized sample spreading method may be difficult to reproduce such phenomena.
The invention creatively provides a dynamic and static combined simulation test resource library construction method to realize high-real dynamic scene simulation, and firstly, a typical static test scene library with test value at each level and function is constructed by utilizing a combined reasoning method and scene screening rules based on a complex scene group formed by the arrangement and combination of the relative positions and the motion relations of all the tested vehicles and all surrounding interference vehicles. On the basis of a typical static test scene library, an online drainage system based on multiple platforms such as webpages, android and the like is developed, the driving right of the interference vehicle around the tested vehicle is taken over through the intervention of online game players, and the real dynamic scene simulation is realized through the participation of real users. And further extracting real driving data in a targeted test scene, training an intelligent traffic model based on artificial intelligent algorithms such as deep learning and the like, and supplementing/replacing real players in the online drainage system. The invention systematically designs a highly real automatic driving simulation system by taking a dynamic and static combined simulation test resource library as a core, wherein a test environment is constructed based on a game engine such as a fantasy 4 engine, and a platform can record all fine performances of a tested vehicle after the tested vehicle starts from a starting point during the test and evaluate the performance of an algorithm from the aspects of vehicle intelligence, driving safety, driving comfort and traffic coordination. The invention well solves the problem of behavior distortion of traffic participants in the current dynamic scene simulation process, greatly improves the test reliability of the simulation system and assists the development of automatic driving technology in China.
Disclosure of Invention
The invention aims to solve the technical problem of providing an automatic driving simulation system so as to solve the problem of behavior distortion of traffic participants in the dynamic scene simulation process of the existing simulation system and improve the test reliability of the simulation system.
The technical scheme adopted for solving the technical problems is as follows: an automatic driving simulation system is designed, a test environment is built based on a game engine such as a fantasy 4 engine, a dynamic and static combined simulation test resource library construction method is provided, high-real dynamic scene simulation is realized, and finally an automatic driving system performance comprehensive evaluation system is built by combining domestic and foreign automatic driving evaluation experience and L1-L3 level automatic driving test characteristics. The invention provides a dynamic and static combined simulation test resource library construction method, which is characterized in that firstly, based on a complex scene group formed by the arrangement and combination of the relative positions and the motion relations of all the tested vehicles and all surrounding disturbance vehicles, a typical static test scene library with test value at each level and function is constructed based on a combined reasoning method and scene screening rules. On the basis of a typical static test scene library, an online drainage system based on multiple platforms such as webpages, android and the like is developed, the driving right of the interference vehicle around the tested vehicle is taken over through the intervention of online game players, and the real dynamic scene simulation is realized through the participation of real users. And further extracting real driving data in a targeted test scene, training an intelligent traffic model based on artificial intelligent algorithms such as deep learning and the like, and supplementing/replacing real players in the online drainage system. After the automatic driving algorithm is connected with the simulation platform and starts to test, the platform can record all fine performances of the tested vehicle after the tested vehicle starts from the starting point, and evaluate the performance of the algorithm in aspects of vehicle intelligence, driving safety, driving comfort and traffic coordination. The invention designs a highly real automatic driving simulation system by taking a dynamic and static combined simulation test resource library construction method as a core system, well solves the problem of behavior distortion of traffic participants in the current dynamic scene simulation process, and greatly improves the test reliability of the simulation system.
The basic principle of the simulation technology is that obstacles, pedestrians, vehicles, weather and the like in the real physical world are simulated in a virtual scene, the obstacles, the pedestrians, the vehicles, the weather and the like are used as sensor sensing objects to test a sensing algorithm, the moving track of traffic participants such as the pedestrians and the like and relevant scene arrangement are simulated to test a decision planning algorithm, and the dynamics characteristics of the vehicles are simulated to test a control algorithm. The automatic driving simulation technology comprises the steps of constructing a test environment, constructing a test scene library, simulating a dynamic scene and establishing an automatic driving system evaluation system.
The test environment constructed should generally include roads (lane lines and pavement materials, etc.), traffic elements (traffic lights and traffic signs), traffic participants (automobiles, non-automobiles and pedestrians), road perimeter elements (including street lamps and buildings), and the like. The invention builds the test environment based on the game engine, can simulate road traffic scene and virtual city scene with extremely high precision, and restore lighting conditions such as refraction, reflection and the like and weather conditions such as wind, frost, rain, snow and the like by means of the game engine, so that the test conditions of scene and sensor simulation are close to reality, and the reality of the test effect of the perception algorithm is ensured. In addition, the physical engine arranged in the game engine can be directly used for simulating the effects of vehicle slip, collision and the like, and support is provided for testing a control algorithm.
The test scene is the comprehensive reflection of the environment and the driving behavior in a certain time and space range, and describes the external states of roads, traffic facilities, meteorological conditions, traffic participants and the like, and the information of the tested driving task and the state and the like. An organic collection of multiple scenes is a "scene library". The autopilot test scenario library is a database consisting of a series of autopilot test scenarios that meet certain test requirements. The invention is based on a complex scene group formed by the arrangement and combination of the relative positions and the motion relations of all the interfered vehicles around the tested vehicle, and based on a combined reasoning method and a scene screening rule, a typical static test scene library with various levels and functions of test value is constructed. The specific steps of the combined reasoning method are as follows:
Determining complex scene groups: for a specified road traffic environment, analyzing the combination of possible relative positions and movement directions of the tested vehicle and N surrounding traffic participants, and determining the relative position ranges of the tested vehicle and the N traffic participants on the premise of covering the most complex combination of driving tasks or the combination which possibly causes traffic accidents;
Screening test scenes: the scene screening rule is designed, and scenes with test value are screened from the complex scene groups to form scene groups such as normal driving working conditions, pre-collision working conditions and the like;
determining parameters: adding necessary constraint conditions into the screened test scene group, and covering all the combined scene groups.
In view of the fact that test scenes in a complex scene group obtained by greedy combination are not all effective, the cost of the test is increased by the unrealizable scenes or the scenes with repeated utility, the invention also needs to reject useless scenes in the complex scene group according to the following scene screening rules:
Separating from the actual scene, and deleting;
Deleting the scene with smaller influence on the tested motion in the similar scenes;
only 1 scene is reserved for the scenes with the same influence on the tested motion.
The dynamic scene mainly refers to the parts of management and control, traffic flow and the like with dynamic characteristics in simulation, is a key component of a simulation test scene, and mainly comprises: traffic management control simulation, motor vehicle/non-motor vehicle/pedestrian motion planning and feedback simulation and the like. Most of the existing dynamic simulation scene simulation systems have the problems that simulation traffic participants lack interaction with a tested vehicle in the simulation test process, and deep research on a generalization method of a real scene is lacking. The online drainage system is designed, an online game client based on multiple platforms such as webpages and android is developed, tests are paid for social and scientific research institutions and government departments, and real traffic interaction is achieved through participation of real users. Furthermore, real driving data are acquired through the background, generalized behavior models are extracted based on artificial intelligent algorithms such as a deep neural network and the like to form a dynamic traffic model library, the model library calls intelligent traffic body models in corresponding scenes to take over corresponding interference vehicles and pedestrians during testing, and the respective behavior modes are corrected through interaction specificity with real traffic participants to achieve high anthropomorphic. The dynamic traffic model library and the static test scene library together form the core of the invention, namely the dynamic and static combined simulation test resource library, and support is provided for high-reality simulation in a typical test scene.
The automatic driving simulation platform can record all fine performances of the tested vehicle after the tested vehicle starts from the starting point during the test period, and evaluate the algorithm performance of the tested vehicle according to the fine performances. The automatic driving system performance comprehensive evaluation system constructed by the invention comprises four aspects of vehicle intelligence, driving safety, driving comfort and traffic coordination, and obtains a comprehensive performance evaluation value based on a multi-level fuzzy comprehensive evaluation method. The performance index system constructed based on the empirical method is as follows:
vehicle intelligence: evaluating the measured autopilot algorithm from aspects of perception, path planning, control and the like, wherein the perception aspect comprises recognition requirements on objects such as signal lamp phases, pedestrians and the like; the path planning aspect comprises the requirements on the aspects of stable track, obstacle avoidance and the like; the control aspect is the requirement of the execution of the planning result, such as maximum deviation, oscillation times, oscillation period and the like;
driving safety: whether the tested vehicle runs on the road can cause danger to other traffic participants, whether the tested vehicle stops in a stop line when in red light, whether the tested vehicle collides with pedestrians, whether traffic accidents occur or not, and the like;
Driving comfort: the driving experience and the interaction experience of the driver during the running of the tested vehicle on the road. During the test, the driving experience evaluates whether the driving of the vehicle is stable or not and whether the turning is smooth or not according to the states of the accelerator, the brake and the steering recorded by the platform in the driving process; the interactive experience is that the interactive experience such as the operation quality of the tested automatic driving system is determined in a questionnaire mode through the fact that the driver is in the loop;
Traffic coordination: the tested vehicle is driven on the road and has traffic movement performance relative to other traffic participants. During the test, the traffic coordination is evaluated by indexes such as running time, traffic flow, road resource effective utilization rate and the like.
After the performance index system is constructed, the overall performance of the automatic driving algorithm is determined based on a multi-level fuzzy comprehensive evaluation method:
(1) Index type unification processing
The qualitative index is quantitatively expressed as a maximum value index;
The quantitative index (maximum, minimum, middle and interval) is converted into a maximum index;
(2) Non-dimensionality treatment of index
The index can be weighted and synthesized after dimensionless treatment, and a normalization treatment method is adopted:
x ij represents the measured value of the jth index at the ith test.
(3) Determination of index weights
Determining the weight of each index by adopting a sequence relation analysis method based on a function driving principle:
sequentially selecting the most important indexes from the index set { x 1,x2,...,xj,…,xm } according to a user-defined evaluation criterion;
Determining a ratio r j(wj-1/wj of importance degrees between the evaluation indexes x j-1 and x j according to table 1;
calculating weight coefficients w j corresponding to the sorted indexes:
wj-1＝rjwj，j＝m，m-1，...，3，2 (3)
table 1r j assignment table
(4) Multi-level fuzzy comprehensive evaluation
On the basis of index system determination, single factor judgment is carried out on a single factor (a certain layer f evaluation index) x j (j=1, 2,.. M, m is the index number), so that membership degree L jk between the single factor judgment and evaluation grade y k (k=1, 2,.. H, h is the grade number) is obtained, and then an evaluation matrix L is formed:
determining a weight set W= (W 1,w2,...,wj,...,wm) based on a sequential relation method, and synthesizing the weight set W= (W 1,w2,...,wj,...,wm) with an evaluation matrix L to obtain a fuzzy comprehensive evaluation model C:
Starting calculation from the lowest layer, forming an evaluation matrix of the upper evaluation index by the obtained result, and iterating to obtain a total evaluation model C t;
Determining a score set μ= (μ 1,μ2,...,μh)T, and calculating a composite score G:
G＝100Ctμ＝(c1,c2,...,ch)·(μ1,μ2,...,μh)T×100 (6)
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects: the game engine is supported, so that the simulation test environment is more lifelike, and the vehicle dynamics simulation is simpler and more convenient. And constructing a dynamic-static combined simulation test resource library, constructing a static test scene library with each level and function of test value based on a combined reasoning method and scene screening rules, and simultaneously realizing real traffic interaction through participation of real players. And further, collecting real driving data through a background, driving intelligent traffic body model training, and constructing a dynamic traffic model library to provide support for AI player driving behaviors. Finally, an automatic driving system performance comprehensive evaluation system is constructed by combining domestic and foreign automatic driving evaluation experience and L1-L3 level automatic driving test characteristics, and comprehensive and reasonable evaluation is provided for algorithm test. Therefore, the method has great significance for improving the overall performance of the automatic driving simulation of the vehicle.
Drawings
FIG. 1 is a schematic diagram of a dynamically and dynamically combined autopilot simulation system architecture;
FIG. 2 is a schematic diagram of a dynamic and static combined test resource library construction;
FIG. 3 is a graph of an autopilot system performance index system;
fig. 4 is a graph of an autopilot system performance index system.
Detailed Description
The invention will be further illustrated with reference to specific examples.
It is to be understood that these examples are illustrative of the present application and are not intended to limit the scope of the present application. Furthermore, it should be understood that various changes and modifications can be made by one skilled in the art after reading the teachings of the present application, and such equivalents are intended to fall within the scope of the application as defined in the appended claims.
In the embodiment of the invention, the framework of the dynamic and static combined automatic driving simulation system and the construction of the test resource library thereof are respectively shown in fig. 1 and fig. 2. In fig. 1, the system framework is divided into four layers including a bottom layer support, a core module, a simulation platform and an evaluation system. The game engine in the bottom support, such as the illusion 4 engine, is used for constructing the simulation test environment, the machine learning libraries of TensorFlow, torch7 and the like are used for supporting the traffic agent model training in the dynamic traffic model library, the databases of MySQL and the like support the driving data collection of players, the test scene storage and the model training, ROS and the like are used as the communication mechanism of each module in the simulation system, and finally the online drainage system is developed based on the platforms of android and the like. Based on the bottom layer support, a core module-dynamic and static combined simulation resource library is constructed, wherein the resource library comprises a static test scene library and a dynamic traffic model library. On the core module, an analog simulation server and a multi-platform user based on platforms such as android, web pages and the like are developed. And finally, establishing a set of evaluation system aiming at various test data of the tested vehicle recorded by the automatic driving platform.
In this embodiment, the construction scheme of the dynamic and static combined simulation resource library provided by the invention is specifically shown in fig. 2. Firstly, based on a complex scene group formed by the arrangement and combination of the relative positions and the motion relations of all the tested vehicles and all surrounding disturbance vehicles, a typical static test scene library with test value of each level and function is constructed based on a combined reasoning method and scene screening rules. As shown in fig. 3, the embodiment uses the straight behavior example of the tested vehicle in the double-lane straight-going section scene to illustrate the static test scene library construction method based on the combined reasoning method and the scene screening rule. In the scenario shown in fig. 3, the presence of the interfering vehicles V1 to V3 has little influence on the running of the test vehicle T, and is not considered. Therefore, only three movement directions (straight, left turn and brake) of the disturbance vehicles V4 and V5 need to be considered, and meanwhile, the situation that V4 or V5 does not exist exists, and 12 test scenes formed by combining the disturbance vehicles V4 and V5 with straight running of a tested vehicle are shown in the following table.
Table 2 straight-going test scene of tested vehicle in double-lane straight-going section scene
When the combination is arranged, the interference-free vehicles are eliminated, so that the second and third test scenes in the scene screening rules are considered, and the 12 test scenes in the table 2 are found to be out of the screening rules, so that all the test scenes are reserved. Then, determining test scene parameters by referring to the existing test rules of L1 level automatic driving, such as EURO-NCAP, and utilizing a game engine to realize corresponding test scenes to store the test scenes in a static test scene library.
And then, a multi-platform user terminal based on platforms such as android, webpages and the like is opened to the society, scientific research institutions and government departments for paid test, and real traffic interaction is realized through participation of real users. And further, collecting real driving data through a background, extracting generalized behavior models based on artificial intelligent algorithms such as a deep neural network and the like to form an intelligent traffic model library under a corresponding scene, calling the intelligent traffic model under the corresponding scene by the model library during the test to take over corresponding disturbance vehicles and pedestrians, and correcting respective behavior modes through interaction specificity with real traffic participants so as to realize high anthropomorphic.
The automatic driving simulation platform can record all fine performances of the tested vehicle after the tested vehicle starts from the starting point during the test period, and evaluate the algorithm performance of the tested vehicle according to the fine performances from the aspects of vehicle intelligence, driving safety, driving comfort and traffic coordination. The present embodiment establishes an automatic drivability evaluation index system as shown in fig. 4. And then determining the overall performance of the automatic driving algorithm based on a multi-level fuzzy comprehensive evaluation method, wherein the detailed steps are as follows:
(1) Index type unification processing
The qualitative index is quantitatively expressed as a maximum value index;
The quantitative index (maximum, minimum, middle and interval) is converted into a maximum index;
(2) Non-dimensionality treatment of index
Carrying out dimensionless treatment on the index by adopting a normalization treatment method;
(3) Determination of index weights
Determining the weight of each index by adopting a sequence relation analysis method based on a function driving principle:
Sequentially selecting the most important indexes from the index set { x 1,x2,...,xj,...,xm } according to a user-defined evaluation criterion;
Determining a ratio r j(wj-1/wj of importance degrees between the evaluation indexes x j-1 and x j according to table 1;
Calculating weight coefficients w j corresponding to the sorted indexes;
(4) Multi-level fuzzy comprehensive evaluation
On the basis of the determination of an index system, performing single-factor judgment on a single factor (a certain layer f evaluation index) x j (j=1, 2,.. M, m is the index number) to obtain a membership degree L jk between the single factor and an evaluation grade y k (k=1, 2,.. H, h is the grade number), and then forming an evaluation matrix L;
And determining a weight set W= (W 1,w2,...,wj,...,wm) based on a sequential relation method, and synthesizing the weight set W= (W 1,w2,...,wj,...,wm) with an evaluation matrix L to obtain a fuzzy comprehensive evaluation model C. Starting calculation from the lowest layer, forming an evaluation matrix of the upper evaluation index by the obtained result, and iterating to obtain a total evaluation model C t;
the score set μ= (μ 1,μ2,...,μh)T) corresponding to the evaluation level was determined, and then the composite score 6 was calculated.
It is to be understood that various other embodiments of the present invention may be made by those skilled in the art without departing from the spirit and scope of the invention, and that various changes and modifications may be made in accordance with the invention without departing from the scope of the invention as defined in the following claims.
Claims (1)
1. The method is characterized by comprising the steps of constructing a test environment based on a game engine, constructing a dynamic and static combined simulation test resource library, constructing an automatic driving system performance comprehensive evaluation system, wherein the simulation test resource library comprises a static test scene library and a dynamic traffic model library, firstly, constructing the static test scene library based on combined reasoning, then introducing a real player into the test system through an on-line drainage system, taking over the surrounding interference vehicles of the tested vehicle and the pedestrian traffic participation objects to realize real dynamic scene simulation, simultaneously extracting real driving data under the test scene, training an intelligent traffic model based on a deep learning artificial intelligent algorithm to construct a dynamic traffic model library, supplementing/replacing real players in the on-line drainage system, and obtaining a comprehensive performance evaluation value based on a multi-level fuzzy comprehensive evaluation method;
The automatic driving simulation system records the fine performance of the tested vehicle after the tested vehicle starts from the starting point during the test period, and evaluates the performance of the algorithm in the aspects of vehicle intelligence, driving safety, driving comfort and traffic coordination;
The specific process of constructing the static test scene library based on the combined reasoning is to construct a typical static test scene library with test value at each level and function by using a combined reasoning method and scene screening rules based on a complex scene group formed by the arrangement and combination of the relative positions and the motion relations of all the tested vehicles and all surrounding interference vehicles;
The combined reasoning method specifically comprises the following steps: determining complex scene groups, screening test scenes and determining parameters;
The vehicle intelligence evaluates the automatic driving algorithm from the aspects of perception, path planning and control, the driving safety refers to whether other traffic participants are dangerous when the tested vehicle runs on a road, the driving comfort refers to driving experience and interaction experience of a driver during the running of the tested vehicle on the road, and the traffic coordination refers to traffic movement performance of the tested vehicle relative to other traffic participants during the running of the tested vehicle on the road;
wherein the determining complex scene group: analyzing the tested vehicles and the surrounding for the specified road traffic environment The combination of the relative positions and the movement directions of the individual traffic participants is used for determining the tested vehicles and/>, on the premise of covering the most complex combination of driving tasks or the combination causing traffic accidentsA range of relative positions of the individual traffic participants;
The screening test scene: the scene screening rule is designed, and scenes with test value are screened out from the complex scene group to form a normal driving working condition scene group and a pre-collision working condition scene group;
the determination parameters are as follows: adding constraint conditions into the screened test scene groups, and covering all the combined scene groups;
The scene screening rule specifically comprises the following steps:
(1) Separating from the actual scene, and deleting;
(2) Deleting the scene with smaller influence on the tested motion in the similar scenes;
(3) The scenes with the same influence on the tested vehicle movement are reserved only by 1 scene;
The system comprises a dynamic traffic model library, a dynamic traffic model library and a learning algorithm-based driving simulation platform, wherein the online drainage system is a multi-platform client developed based on a smart phone, a webpage and a driving simulator platform, a player logs in an automatic driving simulation platform server through a network and takes over traffic participation objects including interference vehicles and pedestrians around a tested vehicle, the dynamic traffic model library is a traffic model set 'scene-intelligent body model library' corresponding to all test scenes in a static test scene library, and elements in the set are traffic participant models driven based on a learning algorithm under the corresponding scenes;
wherein the perception aspect comprises the recognition requirement of objects including signal lamp phase and pedestrians; the path planning aspect comprises the aspect requirements of stable track and obstacle avoidance; the control aspect is the requirement of the execution of the planning result;
the driving safety refers to whether a tested vehicle stops in a stop line when a red light is generated, whether a pedestrian is impacted or not, and whether a traffic accident occurs or not, and the driving comfort refers to whether the driving experience is stable or not and whether the turning is smooth or not according to the throttle, brake and steering states recorded by a platform in the driving process during the test; the interactive experience is determined in the form of a questionnaire;
Wherein, the traffic coordination refers to the evaluation of the traffic coordination through indexes including the running time, the traffic flow and the effective utilization rate of road resources during the test.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581488.7A CN111841012B (en) | 2020-06-23 | 2020-06-23 | Automatic driving simulation system and test resource library construction method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581488.7A CN111841012B (en) | 2020-06-23 | 2020-06-23 | Automatic driving simulation system and test resource library construction method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111841012A CN111841012A (en) | 2020-10-30 |
| CN111841012B true CN111841012B (en) | 2024-05-17 |
Family
ID=72988411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010581488.7A Active CN111841012B (en) | 2020-06-23 | 2020-06-23 | Automatic driving simulation system and test resource library construction method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111841012B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286206B (en) * | 2020-11-17 | 2024-01-23 | 苏州智加科技有限公司 | Automatic driving simulation method, system, equipment, readable storage medium and platform |
| CN112464461B (en) * | 2020-11-20 | 2021-09-28 | 北京赛目科技有限公司 | Method and device for constructing automatic driving test scene |
| CN112541258A (en) * | 2020-12-08 | 2021-03-23 | 特路（北京）科技有限公司 | Test scene library of automatic driving automobile test field |
| CN112732671B (en) * | 2020-12-31 | 2022-08-09 | 华东师范大学 | Space-time trajectory data-driven automatic driving safety scene element modeling method |
| CN112859810A (en) * | 2021-01-13 | 2021-05-28 | 自行科技(武汉)有限公司 | ADAS algorithm verification method and device based on Carla platform |
| CN114813157A (en) * | 2021-01-29 | 2022-07-29 | 华为技术有限公司 | Test scene construction method and device |
| CN112686979B (en) * | 2021-03-22 | 2021-06-01 | 中智行科技有限公司 | Simulated pedestrian animation generation method and device and electronic equipment |
| CN113032261B (en) * | 2021-03-23 | 2022-09-30 | 北京三快在线科技有限公司 | Simulation test method and device |
| CN113052118A (en) * | 2021-04-07 | 2021-06-29 | 上海浩方信息技术有限公司 | Method, system, device, processor and storage medium for realizing scene change video analysis and detection based on high-speed dome camera |
| CN113064839B (en) * | 2021-06-03 | 2021-08-31 | 中智行科技有限公司 | System evaluation method and device |
| CN113377663B (en) * | 2021-06-24 | 2021-12-10 | 吉林大学 | Automatic driving simulation test method based on scene element removing characteristics |
| CN113536548B (en) * | 2021-06-29 | 2024-06-18 | 的卢技术有限公司 | Carla-based construction method of prefabricated track simulation scene |
| CN113609016B (en) * | 2021-08-05 | 2024-03-15 | 北京赛目科技股份有限公司 | Method, device, equipment and medium for constructing automatic driving test scene of vehicle |
| CN113610166B (en) * | 2021-08-10 | 2023-12-26 | 吉林大学 | Method for establishing test scene library for intelligent vehicle |
| CN113867315B (en) * | 2021-09-24 | 2023-06-02 | 同济大学 | Virtual-real combined high-fidelity traffic flow intelligent vehicle test platform and test method |
| CN114065549B (en) * | 2021-11-30 | 2023-04-07 | 广州文远知行科技有限公司 | Automatic driving level evaluation method, device, equipment and storage medium |
| CN114355864B (en) * | 2021-12-28 | 2024-05-03 | 重庆长安汽车股份有限公司 | Model-in-loop test method, system and computer readable storage medium for intelligent driving model development |
| CN115080388B (en) * | 2022-06-08 | 2024-06-25 | 中国科学院软件研究所 | Automatic driving system-oriented simulation test scene generation method |
| CN115688484B (en) * | 2022-11-30 | 2023-07-25 | 西部科学城智能网联汽车创新中心(重庆)有限公司 | V2X simulation method and system based on WebGL |
| CN116302364B (en) * | 2023-05-17 | 2023-08-15 | 清华大学 | Automatic driving reliability test method, device, equipment, medium and program product |
| CN116680932B (en) * | 2023-07-27 | 2023-11-21 | 安徽深信科创信息技术有限公司 | Evaluation method and device for automatic driving simulation test scene |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140144921A (en) * | 2013-06-12 | 2014-12-22 | 국민대학교산학협력단 | Simulation system for autonomous vehicle using virtual reality |
| GB201709348D0 (en) * | 2017-06-13 | 2017-07-26 | Kompetenzzentrum-Das Virtuelle Fahrzeug | Method and process for co-simulation with virtual testing of real environments with pedestrian interaction |
| CN107727411A (en) * | 2017-10-30 | 2018-02-23 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle test and appraisal scene generation system and method |
| CN109657355A (en) * | 2018-12-20 | 2019-04-19 | 安徽江淮汽车集团股份有限公司 | A kind of emulation mode and system of road vehicle virtual scene |
| CN109765060A (en) * | 2018-12-29 | 2019-05-17 | 同济大学 | A kind of automatic driving vehicle traffic coordinating virtual test system and method |
| CN109902430A (en) * | 2019-03-13 | 2019-06-18 | 上海车右智能科技有限公司 | Traffic scene generation method, device, system, computer equipment and storage medium |
| CN110007675A (en) * | 2019-04-12 | 2019-07-12 | 北京航空航天大学 | A kind of Vehicular automatic driving decision system based on driving situation map and the training set preparation method based on unmanned plane |
| CN110243610A (en) * | 2019-05-13 | 2019-09-17 | 北京航空航天大学 | A kind of the multidimensional interference automatic Pilot test macro and method of movement and fixed Combination |
| CN110263381A (en) * | 2019-05-27 | 2019-09-20 | 南京航空航天大学 | A kind of automatic driving vehicle test emulation scene generating method |
| CN110275859A (en) * | 2018-03-15 | 2019-09-24 | 明日娱乐欧洲公司 | Artificial intelligence engine on game engine and chip |
| CN110647053A (en) * | 2019-09-19 | 2020-01-03 | 北京智行者科技有限公司 | Automatic driving simulation method and system |
| CN110779730A (en) * | 2019-08-29 | 2020-02-11 | 浙江零跑科技有限公司 | L3-level automatic driving system testing method based on virtual driving scene vehicle on-ring |
| WO2020079066A1 (en) * | 2018-10-16 | 2020-04-23 | Five AI Limited | Autonomous vehicle planning and prediction |
| CN111123920A (en) * | 2019-12-10 | 2020-05-08 | 武汉光庭信息技术股份有限公司 | Method and device for generating automatic driving simulation test scene |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595901A (en) * | 2018-07-09 | 2018-09-28 | 黄梓钥 | A kind of autonomous driving vehicle normalized security simulating, verifying model data base system |
-
2020
- 2020-06-23 CN CN202010581488.7A patent/CN111841012B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140144921A (en) * | 2013-06-12 | 2014-12-22 | 국민대학교산학협력단 | Simulation system for autonomous vehicle using virtual reality |
| GB201709348D0 (en) * | 2017-06-13 | 2017-07-26 | Kompetenzzentrum-Das Virtuelle Fahrzeug | Method and process for co-simulation with virtual testing of real environments with pedestrian interaction |
| CN107727411A (en) * | 2017-10-30 | 2018-02-23 | 青岛慧拓智能机器有限公司 | A kind of automatic driving vehicle test and appraisal scene generation system and method |
| CN110275859A (en) * | 2018-03-15 | 2019-09-24 | 明日娱乐欧洲公司 | Artificial intelligence engine on game engine and chip |
| WO2020079066A1 (en) * | 2018-10-16 | 2020-04-23 | Five AI Limited | Autonomous vehicle planning and prediction |
| CN109657355A (en) * | 2018-12-20 | 2019-04-19 | 安徽江淮汽车集团股份有限公司 | A kind of emulation mode and system of road vehicle virtual scene |
| CN109765060A (en) * | 2018-12-29 | 2019-05-17 | 同济大学 | A kind of automatic driving vehicle traffic coordinating virtual test system and method |
| CN109902430A (en) * | 2019-03-13 | 2019-06-18 | 上海车右智能科技有限公司 | Traffic scene generation method, device, system, computer equipment and storage medium |
| CN110007675A (en) * | 2019-04-12 | 2019-07-12 | 北京航空航天大学 | A kind of Vehicular automatic driving decision system based on driving situation map and the training set preparation method based on unmanned plane |
| CN110243610A (en) * | 2019-05-13 | 2019-09-17 | 北京航空航天大学 | A kind of the multidimensional interference automatic Pilot test macro and method of movement and fixed Combination |
| CN110263381A (en) * | 2019-05-27 | 2019-09-20 | 南京航空航天大学 | A kind of automatic driving vehicle test emulation scene generating method |
| CN110779730A (en) * | 2019-08-29 | 2020-02-11 | 浙江零跑科技有限公司 | L3-level automatic driving system testing method based on virtual driving scene vehicle on-ring |
| CN110647053A (en) * | 2019-09-19 | 2020-01-03 | 北京智行者科技有限公司 | Automatic driving simulation method and system |
| CN111123920A (en) * | 2019-12-10 | 2020-05-08 | 武汉光庭信息技术股份有限公司 | Method and device for generating automatic driving simulation test scene |
Non-Patent Citations (3)
| Title |
|---|
| SAE Level 3 Autonomous Driving Technology of the ETRI;KyoungWook Min.et al;2019 International Conference on Information and Communication Technology Convergence (ICTC);第464-466页 * |
| 构建基于真实场景下自动驾驶车辆模拟仿真测试;吴琼;智能网联汽车;第70-71页 * |
| 陈旭梅.城市智能交通系统.北京交通大学出版社,2013,第239-241页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111841012A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111841012B (en) | Automatic driving simulation system and test resource library construction method thereof | |
| Li et al. | Survey on artificial intelligence for vehicles | |
| Algers et al. | Review of micro-simulation models | |
| Miao et al. | A game-engine-based platform for modeling and computing artificial transportation systems | |
| CN110716529A (en) | Automatic generation method and device for automatic driving test case | |
| Taillandier | Traffic simulation with the GAMA platform | |
| CN113887037A (en) | Automatic driving system evaluation method under network connection environment with different permeability | |
| Roldán-Gómez et al. | SwarmCity project: monitoring traffic, pedestrians, climate, and pollution with an aerial robotic swarm: Data collection and fusion in a smart city, and its representation using virtual reality | |
| Xu et al. | Spatial ensemble prediction of hourly PM2. 5 concentrations around Beijing railway station in China | |
| CN117151246B (en) | Agent decision method, control method, electronic device and storage medium | |
| CN117313931B (en) | Subway platform passenger evacuation time prediction method based on fire scene | |
| Ji et al. | Perspective, survey and trends: Public driving datasets and toolsets for autonomous driving virtual test | |
| Abu-Shawish et al. | Metaheuristic techniques in optimizing traffic control lights: a systematic review | |
| Shi et al. | Serious Games for Transportation Research: An Overview | |
| Chu et al. | Three-dimensional simulation for training autonomous vehicles in smart city environments | |
| CN118015830A (en) | Pedestrian behavior prediction method for signaling-free right-hand fork man-vehicle game | |
| US20230154323A1 (en) | Automatic real time information to vehicular occupants | |
| WO2023123130A1 (en) | Method and apparatus for autonomous driving system, electronic device and medium | |
| Rebling et al. | Towards the Interpretation of Customizable Imitation Learning of Human Driving Behavior in Mixed Traffic Scenarios | |
| Caleffi et al. | Small-scale self-driving cars: A systematic literature review | |
| Pijoan et al. | GTPlat: Geosimulation for Assessing the Application of Incentives to Transport Planning | |
| CN118312426A (en) | Method for generating and optimizing test scene of automatic driving vehicle based on collision point | |
| Li et al. | Game-Generative Adversarial Imitation Learning for Pedestrian Simulation During Pedestrian-Vehicle Interaction | |
| Silva | Traffic lights optimization of an intersection through the application of metaheuristics | |
| Ren et al. | Autonomous Driving Simulator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |