CN110427827A - It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network - Google Patents

It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network Download PDF

Info

Publication number
CN110427827A
CN110427827A CN201910607644.XA CN201910607644A CN110427827A CN 110427827 A CN110427827 A CN 110427827A CN 201910607644 A CN201910607644 A CN 201910607644A CN 110427827 A CN110427827 A CN 110427827A
Authority
CN
China
Prior art keywords
network
avoidance
feature
perception
driving
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910607644.XA
Other languages
Chinese (zh)
Inventor
张海涛
康瀚隆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Liaoning Technical University
Original Assignee
Liaoning Technical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Liaoning Technical University filed Critical Liaoning Technical University
Priority to CN201910607644.XA priority Critical patent/CN110427827A/en
Publication of CN110427827A publication Critical patent/CN110427827A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Chemical & Material Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Analytical Chemistry (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network, primarily focus on the deeper exploration to sensing capability and effective promotion to abilities such as avoidance planning, network interpretations, and be three weaker big key problems described above, propose a kind of effective solution scheme.Network structureization is layered first to improve the interpretability of network parts, secondly by pre-training FPN network [15] to improve the multiple dimensioned sensing capability of network, and increase obstacle avoidance module to optimize vehicle planning ability, eventually by avoidance feature and planning navigation instruction control control flow direction is perceived, to realize vehicle to the effective Feedback of global intention and local feature.

Description

It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network
Technical field
The present invention relates to a kind of unmanned faulty analyses of autonomous driving system navigator fix process and safety assurance technology neck Domain in particular, provides the autonomous driving network under the multiple dimensioned perception of one kind and Global motion planning.
Background technique
From nineteen fifty, Barret Electrnies company, the U.S. develops first autonomous driving automobile and rises, autonomous driving Technology becomes as the research direction of automotive field hottest point.Behind the addition automatic Pilot camp in 2009 Google, automatic Pilot Technology becomes the strong chip of major high-tech company science and technology competition soon, and is proposed Waymo, Cruise successively, Apollo, Zoox, Nuro etc. outstanding autonomous driving automobile[1].These autonomous driving systems are unified according to perception, planning and control Molding block is designed, and is emulated by a large amount of clouds and carried out tuning.Although these leading autonomous driving systems have higher Tuning effect, but part bottleneck is still remained, such as under controlled calculating force constraint, the bottom Semantic Aware shown is not It determines, high-level semantics perceive the problems such as unstable and planning mode tends to regularization.And this structure system is to hsrdware requirements Excessively high, such as 64 line velodyne laser radars are individually just worth tens of thousands of U.S. dollars, and ultrasonic wave, millimetre-wave radar needed for other It is all high cost with a large amount of mono-/bis-mesh camera, and these are also the expense of sensor, there are no computing system works The high system costs such as control.Such autonomous driving system due to needing to consider many levels such as timeliness, cost, safety, Cause many ripe algorithms all since the limitation of hardware computing capability can not finally put into real-time application scenarios.And another base In the end-to-end autonomous driving strategy of neural network, proposed from 1988[2], just greatly pursued, and in 2016 years After the end-to-end autonomous driving system publication of NVIIDA DAVE-2[3], it is increasingly becoming and quickly and efficiently builds automatic Pilot system in recent years The development trend of system.End-to-end autonomous driving network perceives the mapping function of control by training[2-7]It realizes end to end certainly Main driving ability had not only abandoned high-cost system architecture, but also solved the problems such as perception, control plane coordination is inconsistent.This Automatic Pilot strategy end to end is planted, just feature succinct with itself, highly efficient, unified, becomes autonomous driving research field gradually The research direction of rise.
Currently advanced end-to-end autonomous driving system mainly includes NVIDIA DAVE-2 network[3]With Felipe et al. in 2018 propose based on higher level constraints instruction under autonomous driving network[4].Wherein DAVE-2 network relies on learning by imitation rate First mankind's driving behavior is learnt using end-to-end neural network, final Recurrent networks driver behavior, is realized substantially autonomous Driving ability, but still remain following problems:
(1) ability for lacking Global motion planning does not determine driving direction at traveling to crossing, can not be according to internal system Driving intention complete corresponding behavior, only can reappear trained driver as far as possible and travel to the driver behavior behind current crossing, Corresponding selection can not be made；
(2) together, the design of excessively black box leads to not be returned according to error sample for perception, planning, control layer close coupling Return wrong origin, also can not carry out hierarchical algorithm optimization by modifying each layer effect；
(3) sensing capability is excessively weak, and the characteristic area of its visible study concentrates on after by network model deconvolution The boundary on road and the relatively large object of surrounding but ignore more it should be noted that the structured objects information such as vehicle, pedestrian[7]；
(4) lack obstacle avoidance ability, network is difficult to learn the brake of avoidance or turns to behavior.
Document[4]Then on the basis of problem (1), it is conceived to Global motion planning for the impact effect of vehicle control end, tentatively divides From perception and control layer, and the control under special scenes is constrained by global high level instructions and is fed back, it is complete under such as crossroad The direction of office's planning, using high level instructions as the screening washer between control branch to realize the different controls under different high level instructions Effect.Although the realization of this network largely solves study unicity and Global motion planning missing of autonomous land vehicle etc. Problem[4], but effective solution still is not proposed to problem (2-4), autonomous driving network still lacks explanatory, scarce The weary learning ability to avoidance problem, shortage Multi-scale model sensing capability, and learning characteristic is also still laid particular emphasis on Roadside circle and be short of the learning method to important structured objects (automobile, pedestrian etc.).
Summary of the invention
The object of the present invention is to provide the autonomous driving networks under a kind of multiple dimensioned perception and Global motion planning.Spectrum enhancing effect Fruit is good, checks that precision is high.
The technical solution adopted is that:
It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network, first by network structureization be layered to improve net The interpretability of network each section secondly by pre-training FPN network [15] to improve the multiple dimensioned sensing capability of network, and increases Obstacle avoidance module is flowed to optimizing vehicle planning ability eventually by perception avoidance feature and planning navigation instruction control control, with Realize vehicle to the effective Feedback of global intention and local feature.
1. the multiple dimensioned sensing module based on FPN
It is high semantic by successively feedforward convolution algorithm acquisition top layer to extract the stage for end-to-end autonomous driving network characterization at present Feature.The characteristic extraction procedure realized even so has certain sensing capability to high semantic feature, but since it is carried Characteristic dimension size it is limited, so currently advanced autonomous driving network to Analysis On Multi-scale Features and do not have good perception effect Fruit.And it, can not be to structured objects (vehicle, pedestrian etc.) due to the end-to-end learning by imitation mode of current autonomous driving network Effectively distinguished, cause to focus mainly on road boundary in network characterization extraction process and the structuring of ignoring no less important Such as target, the similarly design due to network black box itself make current autonomous driving network lack the channel of layering tuning, i.e., There is exception control as a result, can not determine it is which, there are problems in stage in fruit network, can only increase strengthening training simply, not There is preferable prioritization scheme.And herein then in view of the above problems, use for reference currently advanced multiscale target detection method, and combine Object detection area regression scheme, the final perception for realizing network to Multi-scale model target, and area is provided for sensing network The explanatory channel in domain.It will be amplified from conventional target detection method to multiscale target detection method, and taken on this basis herein Sensing network is built, network is explained eventually by provincial characteristics, increases sensing network region interpretation approach, complete to multiple dimensioned perception The whole design of network.
Traditional CNN (Convolutional Neural Network) target detection feature extraction process, such as Faster RCNN[11], it only depends on and top-level feature is predicted, although high-level characteristic possesses richer semantic information, at this time Feature locations Relative Fuzzy and the particularity due to upper sampling process, cause many small scale features in top-level feature figure It is very little.Even if similar SSD network[14]To solve the problems, such as multiple dimensioned extraction, carried out using the method for different characteristic figure pre- It surveys, solves the problems, such as part multiple scale detecting, but still have ignored the feature of the small scale of many bottoms.Such as Fig. 1 (a) institute Show, traditional feature prediction only considers the semantic feature of top layer P3, but more fully information many for P1, P2 layers is but neglected Slightly, lead to not effectively extract multiple dimensioned feature.If organically combined by some way it is assumed that P1, P2, P3 are adopted Get up, not only comprising high-level semantics feature but also include multiple dimensioned rudimentary semantic feature, can effectively solve the problems, such as this, and FPN (Feature Pyramid Networks) network[15]Proposition be namely based on such a it is assumed that as shown in Fig. 1 (b), CNN Network is no longer on linear structure, and becomes the feature pyramid structure of top-down lateral connection, and every layer of characteristic pattern is all and together The association of scale feature figure, each scale feature figure can all predict different characteristic, reach effective extraction and benefit to Analysis On Multi-scale Features With.FPN network realizes comprehensive concern to low-level image feature by the fusion to same scale characteristic pattern, makes it to multiscale target Reach good recognition effect.
FPN network as shown in formula 1-2, whereinFor n-th of characteristic pattern after fused, g is FPN structure, and v is then two-wire Property interpolating function, fm×mThen correspond to the convolution operation carried out from the convolution kernel of different m × m sizes.φnThen character pair extracts net Network n-th layer characteristic pattern, I are original image.The FPN network so realized, although can have preferable identification energy to multiple dimensioned Power, but information transfer capacity is weaker between characteristic layer non-conterminous in top-down fusion process, by multilayer After fusion, the influence of farther away characteristic layer adjacent with current layer but becomes very little, so obtaining not in FPN network herein After the characteristic pattern of scale, by the way of unified scale feature fusion, so that characteristic pattern is not solely dependent on single scale, Make sensing layer that there is unified receptive field, reduces the perception information to planning layer output redundancy, the perception net built herein such as Fig. 2 Shown in the Fused module of network.In sensing network is built, herein in conjunction with FPN and PSPnet[16], and in feature pyramid network The top layer of network is added to PPM (Pyramid Pooling Module) pyramid pond module.PPM is the most crucial net of PSPnet Network structure mainly passes through and increases CNN overall situation receptive field, to achieve the purpose that multiple dimensioned pond.Herein most based on above-mentioned theory Multiple dimensioned perception fusion feature figure P10 is obtained eventually.
φn=fn(φn-1)=fn(fn-1(...f1(I))) (2)
Herein on the basis of combining multiple dimensioned perception information, to obtain more structured objects features as far as possible, choose RPN (Region Proposal Network) region candidate network[11]The perceptually interpreter of module, to structured objects It carries out carrying out candidate region extraction, so that sensing layer has structured objects recognition capability and network interpretation ability.RPN network sheet The function that selective search i.e. region is suggested is realized in matter.RPN network is obtained by carrying out sliding window operation to characteristic pattern The area score of each sliding window position and position return revised region and suggest, finally carry out non-maximum suppression to candidate region System, to obtain the testing result after perception interpretation layer understands feature.
As shown in figure 3, herein on the basis of sensing network, joint-detection regression block constructs the judge to sensing results Network provides the channel of detection and tuning for sensing network.And network will be judged as target detection network and carry out structure allelopathic Know target pre-training, realizes that sensing network to the priori understandability of structured features, and is being implanted into end-to-end autonomous driving mould After type, it is trained using perception pre-training parameter as initial parameter, will test recurrence layer, perceptually the interpreter of layer, with This deepens the understandability to sensing network.
Herein in conjunction with FPN, PPM and RPN network, energy is explained to weak multiple dimensioned sensing capability and poor sensing network Power proposes corresponding solution, and then designs autonomous driving sensing network.The sensing network designed even so, increases entirety The workload of network pre-training, but be the study of network autonomous driving in the future, optimization tuning etc., it opens more succinct intuitive Perception shortcut.It is believed that in the course of end-to-end autonomous driving network fast development, this pretreated method of perception can be at For the pre-training mode that autonomous driving network is universal.
2. based on the planning module apart from avoidance
Avoidance problem is always to be difficult the ability of autonomous learning in learning by imitation, but irreplaceable due to module itself Property, so that it is increasingly becoming one of the important indicator for measuring autonomous driving effect superiority and inferiority.Although presently, there are the sides of related avoidance Method[25-27], but it is not provided with the avoidance scheme of depth learning by imitation, accomplish effectively so that nowadays autonomous driving network is more difficult Avoidance, and very big reason is excessive learning parameter, non-structured learning characteristic and without preferably study side among these Caused by method.
The mode of new characteristic present method and study is proposed herein in conjunction with the advantage of learning by imitation, and as control The priori features of module carry out control forecasting.Traditional avoidance problem, as shown in Equation 3, wherein c is to drive vehicle, and d is vehicle peace Full distance, b are barrier, and O is object boundary point set, and i is the quantity of t moment barrier.
D < Ot(bi)-Ot(c) (3)
Although this avoidance mode can ensure the safe driving at each moment, but not consider speed to braking distance The many factors such as influence and driving experience are only applicable to the non-vehicle for multiplying fortune and driving of low speed.
Using speed v as intelligent barrier avoiding reference parameter on the basis of herein, formula 4 can be obtained.
D < Ot(bi)-Ot(c)+(vt(bi)-vt(c))t (4)
Wherein v is that t moment corresponds to the speed of object, if only considering non-retrograde situation, i.e. v at this timet(bi)≤0, it only needs to meet Safe avoidance under fast state can be realized in formula 5, and avoidance mode is also made to become relying on itself shifting speed and observation moment relative distance Mapping function.Since sensing network only extracts the feature at forward sight visual angle, thus obstacle avoidance module increase speed on this basis with The mapping relations of distance, and result is directly acted on into control module.
D < Ot(bi)-Ot(c)+vt(c)t (5)
Herein based on above-mentioned analysis, design avoidance network is as shown in Equation 6, whereinFor apart from fusion feature,For perception Fusion feature, s are current vehicle speed, and fc is full connection operation, and m is full connection dimension.And increases for avoidance planning network and judge mould Block, expanded type 6 obtain formula 7, and wherein d is that avoidance returns distance, and the accuracy of avoidance web results is judged with this.
Based on view of the above, avoidance network as shown in Figure 4 is built herein, according to avoidance mapping relations, to guarantee distance Feature integrality introduces velocity characteristic and the two is merged full connection result and merge with distance feature, makes result maximum journey Under the premise of degree is dependent on the two feature, prior distance feature is not destroyed.It can be special respectively as distance for fusion results Incoming subsequent control module is levied, and obtains and judges distance results.
The advantage being designed in this way is that priori velocity information adjusts the distance prediction and avoidance obstacle all has great importance, and It does not destroy image distance feature on the basis of merging speed, under the premise of guaranteeing obstacle avoidance module to the dependence of the two, makes network More standby the stressing property of training.
3. the control module based on Global motion planning
Since end-to-end autonomous driving network learns by imitation mankind's driving data, lead to the driver behavior finally learnt Place one's entire reliance upon the Driving Scene of acquisition, so when Driving Scene has alternative path (such as crossing), individually from image In can not determine the real driving intention of driver.It then needs to introduce Global motion planning scheme at this time, be advised in internal system according to the overall situation Check off fruit provides planning path, makes autonomous driving network in alternative path, obtains unique planning path, final to solve certainly Main driving network makes the control result for violating system intention in more options path[28]
Document is used for reference herein[4]Control module screening technique, design this paper Global motion planning control network.It calculates first current Position and target position optimum programming path, and to current driving behavior, Global motion planning instruction is provided, each planning instruction corresponds to Difference control branch realizes that network can take correctly control feedback under different demands for control.
As shown in figure 5, conditional order c is instructed as Global motion planning, switching control module branch network is felt on shared upper layer Under the premise of knowing with avoidance information, consistency and uniformity that driving intention flows to control module are realized, herein, this Kind of control module particularly may be divided into along follow, intentions of turning left, turn right be intended to, intention etc. of keeping straight on.
4. end-to-end autonomous driving core network
It herein will be in learning by imitation theoretical basis[24]In conjunction with above-mentioned branching networks, finally to whole autonomous driving trunk Network carries out master-plan and analysis.
Autonomous driving learning by imitation, which is laid particular emphasis on, learns the subjective driving condition of professional driver, imitates each scene The driving behavior and driving purpose of lower driver, finally realizes learning by imitation[29-32].One is set herein in discrete time node T and controller M with environmental interaction.Current observed information o is received in each timing node t, controller MtAnd take correspondence Feedback action at, that is, complete the process of feedback of a controller M.And the core concept of learning by imitation is the driving according to the mankind Such a controller M of Behavioral training makes its feedback action more approach the movement of the mankind.Then training set D is defined, such as 8 institute of formula Show, wherein by the currently active observation oiAnd a is fed back in the currently active movementiConstitute primary effective training sample.It is assumed that the mankind Movement is successfully taking accurate feedback action to field of view every time, that need to define one and can approach as far as possible The mankind feed back the mapping function of behavior, can realize basic learning by imitation, as shown in Equation 9
The mapping function F for wherein defining an imitation controlling behavior, passes through current time oiWith corresponding parameter θ, acquisition pair Answer a at momenti, when mapping function F most approaches mankind's feedback action, can realize the training of learning by imitation.
For perceiving branching networks, sensing results mapping function P is defined, current time o is passed throughi, and corresponding parameter ε, it obtains Take sensing region testing result pi, as shown in Equation 10, in perception branching networks pre-training, only the parameter ε of function P need to be made unlimited Level off to piAnnotation results, that is, complete perception branching networks pre-training.
For avoidance planning network, avoidance planning is defined apart from mapping function B, passes through current time Perception Features vector pfi With velocity vector siAnd corresponding parameter lambda, obtain avoidance distance di, as shown in Equation 11, and due to learning by imitation mapping relations The ability of avoidance relatively difficult to achieve, i.e. vehicle can not be according to sensing results oiMake due avoidance feedback.So herein by speed Vector siAs the output feature that another input is perceived with perception branch outcome collectively as avoidance, as shown in Equation 12.
Network is controlled for Global motion planning, the output result of driver is no longer only dominated by observation result, relied more on In the state of internal system.The vector h of definition characterization internal system state at this time, by itself and observation result o, velocity vector s mono- Play the reference as feedback action, i.e. ai=E (oi,si,hi), to solve the barrier that the instruction of system inside and outside is linked up.Wherein h packet Subjective intention containing user, the state of the internal systems such as system Global motion planning instruction.C is inputted eventually by additional command is introduced =c (h) makes system sneak condition become the important references factor of control instruction, as shown in Equation 13.It is designed in conjunction with overall network, Introductory die is finally imitated into learning training diversity 8 and extends an accepted way of doing sth 14.
The end-to-end automatic Pilot network designed herein, is made of, as shown in Figure 6 sensing layer, planning layer, control layer.Sense Know layer by input Image i, FPN f and Fused module composition.Wherein Image i is to be converted into from camera sensing device reception 300 × 100 RGB image, FPN module then increases PPM structure on the basis of FPN, and the structure is according to PSPnet network design Theory, design PPM are that global average pond layer and 3 × 3 convolutional layers, Fused fused layer are carried out by the way of bilinear interpolation Fusion obtains perception characteristic results eventually by the full articulamentum of 512 dimensions, realizes sensing layer effectively mentioning to Analysis On Multi-scale Features It takes.Perception interpretation layer does not embody in core network, but in training and optimization process, needs to perceive fusion feature figure biography Enter in RPN network, obtains region candidate as a result, sensing network is explained and optimized with this.Planning layer is then respectively by avoidance Module, Fusion Module, planning instruction valve are constituted.Wherein obstacle avoidance module constructs 3 layer of 128 neuron according to avoidance mapping relations Ring type connects obstacle avoidance module entirely, the input using avoidance characteristic results as Fusion Module and the input for returning distance d, to keeping away Barrier result is iterated.Fused layer fusion perception and avoidance feature simultaneously uniformly consign to planning instruction valve c, select corresponding control net Network.Control module is made of four control branching networks, and each control branching networks act a by two full connections and a recurrence It constitutes.Control layer is instructed by Global motion planning, is selected corresponding control module, is established the communications conduit of outer bound pair network, is realized complete Learning by imitation under office's planning.
Detailed description of the invention
Fig. 1 tradition CNN and FPN feature extraction comparison diagram.
Fig. 2 FPN sensing network structure chart.
Fig. 3 detects Recurrent networks structure chart.
Fig. 4 avoidance network structure.
Fig. 5 condition controls network structure.
Fig. 6 autonomous driving core network architecture.
Fig. 7 perceives pre-training network diagram.
Fig. 8 collecting training data schematic diagram.
Fig. 9 autonomous driving network training schematic diagram.
Figure 10 driving task navigation picture.
Specific embodiment
1. experiment flow is analyzed
Experimental stage is first the data set preparation stage, during which need to prepare two parts data, and one is for perceiving pre- instruction Experienced target detection public data collection, secondly for the standard driving data collection for autonomous driving network training；Target is examined Public data collection is surveyed, the optimal screening of network branches need to be carried out under the data set, to find the perception net for being most suitable for the scene Network and detection Recurrent networks, and make the interpreter for detecting Recurrent networks perceptually network, recurrence detection is carried out to Perception Features, And sensing network is also accordingly iterated adjustment to inherent parameters dependent on the optimum results of interpreter, is finally completed sensing network The pre-training of structured objects sensing capability, it is specific as shown in Figure 7.
For driving data collection, as shown in Figure 8, it is necessary first to choose for autonomous driving network data acquisition, test Emulation platform, and selection criteria depends primarily under the emulation platform driving data that whether can acquire demand and with the presence or absence of more Mostly testable Driving Scene；After choosing emulation platform, by according to the method for learning by imitation, standard is acquired in emulation platform Expansible driving data, these expansible driving datas, the main input/output comprising autonomous driving core network, The input/output of avoidance network and corresponding Global motion planning instruct etc.；It is Logistics networks for driving in data acquisition It sails control and is better understood from learning ability, so random interference noise need to be added to it in this process, learnt by confrontation Mode, make network understand control it is original；For the standard driving data collection acquired, data need to be carried out to the data and located in advance Reason, predominantly data scrubbing and Data expansion, to ensure the certain generalization ability of the quality and network of learning data.
For pretreated driving data collection, as shown in figure 9, using multi-task learning mode, to autonomous driving backbone network Network and avoidance network carry out learning training, and optimize the network parameter of its each section by optimizer；Finally in a simulated environment, Driving assessment is carried out to autonomous driving network.
2. multiple dimensioned perception pre-training
Herein firstly, choosing VOC2007 and BDD100K[33]Public data collection as pre-training data set, and at present compared with Mature algorithm of target detection carries out unified training and assessment, obtains table 1.As can be seen from the table, FPN network has road target There is higher detection effect, and mAP (Mean Average Precision) is basically stable at 80%, since the network is higher Verification and measurement ratio and more stable detection effect, so the core network of selection FPN perceptually module herein.
Each network objectives testing result contrast table of table 1
After choosing FPN model, pre-training is carried out in conjunction with RPN network composition target detection network, and obtain sensing results. By test picture as it can be seen that testing result is coincide substantially test set annotation results, and show detection the higher inspection of each scale of network Accuracy rate and recall rate.
The sensing network trained is subjected to layering visualization, it is seen that network for all vehicles have specific perception with Feature understandability, and for low-dimensional characteristic layer, the feature of e-learning more focused on edge local feature, and for For high dimensional feature layer, network is then more focused on whole global characteristics.Layer is returned based on RPN detection herein to carry out sensing network Pre-training, result presentation its had expected priori understandability for structured objects.Finally priori is learnt to join herein The initial parameter that branching networks are perceived when number is as autonomous driving network training, is finally trained.
3. emulating data acquisition
To acquire vehicle driving data, so needing to choose suitable vehicle emulation simulator as data acquisition, test Platform.Autonomous driving emulation simulator is broadly divided into two major classes on technological layer: the first kind is based on generated data, to ring Border and vehicle are simulated, Carla, Carsim, Unity, Panosim etc.；Second class then lay particular emphasis on to playback of data into Row emulation, such as Autoware, Apollo.
Herein due to experiment condition limitation, select first kind simulated environment tested, and select compared to Carsim, The Carla emulation platform that Unity, Panosim are easier to dispose, build is tested.
CARLA[34]For based on virtual engine 4 realize and support in urban environment to automated driving system exploitation, training, The open source emulation platform of test.It include the cities and towns of two Specialty Designs, building, vegetation, traffic sign, vehicle, pedestrian, when Between, the modules such as weather, and be equipped with the Aerodynamic parameter of profession.Relative to other emulation platforms, Carla provides more open City scenarios layout, construction vehicle allot, personnel assignment etc., and Carla provides more fully sensor deployment scheme, all Such as it is single binocular camera, laser radar and corresponding RGB RGBD, PCL.
Since Carla emulator provides two sets of early cities scenes and many configurable test environment, so to ensure The quality of experimental data acquisition, it is prewired to the scene progress of different weather, different periods, different road concentrations respectively herein It sets, by being located in scene as above outside Logitech G29, drive simulating training is carried out, to ensure driver in complex scene Control feedback can correctly be made.Carla is preset into City scenarios as initial training scene to complete a set pattern first herein The training and test assignment of mould；Secondly, being realized for special training, test scene by the environmental simulation to special screne Driving ability of the network under different scenes；For whole testing scheme, herein using to global multitask map nodes It is default to realize distributing for more driving tasks, driving ability and network of the driver under multitask scene are strengthened in multitask with this Learning ability under scene, and then realize and learning by imitation is efficiently used.Due to the driving ability of this paper autonomous driving network It is largely dependent on the driving ability of driver, so still needing during increasing multi-task learning scene for network It wants driver to be rested under various study scenes in advance to vehicle condition, road conditions relatively accurately driving feedback ability, could be received with this Collect the higher driving record of confidence level, and then determines whether network can more accurately understand the drive manner of driver, it is final real Now to effective study of driving ability.After network possesses certain driving ability, can be pressed in Carla herein weather, the period, The test scene in orientation, road concentration setting difference Training scene, detects extensive energy of the network under each scene with this Power.
After driver, which has, stablizes driving ability under emulation platform, Town1 is selected to acquire city as data in Carla City, and driving task to be collected is distributed for driver.The road in cities and towns 1 is divided into different roads as unit of 10 meters herein Node R oadNode.And wherein will be defined as IntersectionNode in crossing, individually additionally marked from RoadNode, That is IntersectionNode ∈ { RoadNode }.The node chosen in RoadNode herein establishes Benchmark learning tasks, And initially study saves in each task of internal state c (h) screening at middle IntersectionNode and corresponding crossing according to the map Point and end study node, realize the mode of learning that difficulty is gradually incremented by.As the Benchmark most started only distributes roadside The simple task of traveling gradually distributes turning crossing and jam road etc. complicated driving task again later.Every group of driving is appointed Business iteration is assigned under each weather scene, so that driving data covers each weather scene, to promote network in different scenes Under adaptability, it is final to be arranged according to above, generate driving task of the multiple groups based on RoadNode, carry out standard for driver Driving data acquisition.
Since the terminus of task is only marked in driving task, this for the driver for being not familiar with town road, Increase certain burden unavoidably, and this situation that noise may be generated to driving data, it ought to evade.So driving herein At the beginning of member's data acquisition, according to driving task terminus, is obtained by dijkstra's algorithm and gone directly the optimal road of terminal by starting point Diameter, and indicated in the figure of cities and towns, as shown in Figure 10, wherein blue circle is current location, red circle is final position, arrow It is identified as the planning direction that will reach crossing.Driver carries out specification driving, to avoid it because excessively examining by planning path The driving noise considering vehicle traveling direction and generating.In driving procedure, driver need to control vehicle to be no more than the speed of 60km/h Metric model traveling, and instructed according to the planning path of driving task and Global motion planning, correctly drive vehicle.When driver travels After to next road circuit node, current vehicle position be will be updated to new node, i.e. in Figure 10, blue circle will be carried out with vehicle driving Location updating.When driver travels to new node and when at a distance of 10 nodal distances of IntersectionNode, Global motion planning refers to C (h) is enabled, i.e. navigational arrows mark in Figure 10 will be updated (straight to the driving intention of nearest IntersectionNode node Row turns left, turns right), driver need to make correct driver behavior according to the instruction, and it is anti-to make corresponding avoidance to avoidance scene Feedback, if occur happening suddenly in driving task or driver behavior mistake when, need driver to pass through the preset key of steering wheel This driving task is restarted, to ensure to acquire the quality of data.Setting bodywork reference frame center herein, pixel are 800 × 600 Monocular cam is as data sampling sensor.Driver controls vehicle by Logitech G29 peripheral hardware steering wheel and pedal It is travelled with the velocity gauge no more than 60km/h.When each acquisition starts, preset driving task group is chosen, and every When a driving task starts, starting, the end node of current task are obtained, optimal path is obtained by Dijkstra, and every It is secondary be driven to next node after, present node and start node are substituted, driver according to Global motion planning instruct and work as Preplanning path carries out specification driving.
The data of acquisition are by sensing data, driving task, planning instruction set, control data, avoidance data, perception number According to, the composition such as system data, emulation platform configuration parameter.Herein using car body front axle center and the vertical line intersection point on ground as referring to Origin establishes bodywork reference frame by right-handed coordinate system, and respectively at setting 800 under (0,0,0.6), (- 2,0,1.6) coordinate × The laser radar sensor of 600 monocular cam and 32 lines.Wherein sensing data, the main acquisition for collecting monocular cam Data；Driving task then records the start node and terminal node of current task；It plans instruction set, records current running state Under planning command status；Data are controlled, currently comprising steer (direction), throttle (throttle), three kinds of brake (brake) Controlled attribute, wherein [- 1,1] steer ∈, throttle ∈ [- 1,1], brake ∈ [0,1], corresponding extreme value correspond to steering wheel It is left and right to make and pedal tramples feedback ratio；Avoidance data, are recorded using laser radar data, with sensor coordinates (2,0, -1) coordinate is reference center under system, in horizontal [- 1,1] distance, determines 3 effective obstacle-avoidance areas by 0.5m step-length, It asks respective distances to be less than within the scope of 20m and puts the average distance of cloud as region actual distance x in each regiont, and press laser thunder Delay time t is set up to frequency is sent, obtains feasible distance x in delay timev, by the two difference xgtJudgement as the region Distance, finally by 3 regional determinations apart from the avoidance criterion distance as the frame；System data, the current traveling of record vehicle State, it is current drive node location and present system time, wherein driving status include automobile speed with whether generate collision, The incorrect driving such as crimping；Emulation platform configuration parameter, then respectively record current driving scene vehicle/crowded degree, Weather conditions drive the configuration parameters such as period.Specific acquisition mode, driver drive rule by navigation information, according to correct Model carries out the driving data acquisition of vehicle, and stores every 10 frames to acquisition data.
Due to the more difficult driving posture for restoring itself in control interference of learning by imitation[29-31], so using for reference text herein It offers[31]Method, to turn to and controlling the interference noise that signal injects random deviating road track, and be its superposition triggering collision Interference noise the vehicle being disturbed is carried out correctly to correct response so that driver is in data acquisition.In data When storage, the interference noises such as offset, collision can't be stored in data by the control feedback result that control data only record driver In.
4. data prediction
For Support Training data accuracy, diversity and scalability the features such as, need driving to collection to acquire number According to the data predictions such as progress data scrubbing, data screening and data enhancing work.
For the data of acquisition, need first to clear up invalid data, such as initially without control output acquisition data, It generates collision/crimping etc. and drives data lack of standardization, with the accuracy of Support Training data control feedback result.
For the data cleared up, need first separate training set according to the ratio of 6:4 and verifying collect, and in respective set into The more category filters of row, classification type are divided by emulation platform preconfiguration parameters, and main purpose is surveyed for more classification later Examination, more scene optimizations are prepared.
For the data screened, Data expansion is carried out to initial data using data enhancing conversion set herein, and fixed Justice conversion collection is combined into contrast, brightness, Gaussian Blur, Gaussian noise, salt-pepper noise etc., random herein for original image data Increase a variety of conversions for it, to achieve the purpose that data enhance, and then the generalization ability of lift scheme.
5. autonomous driving network training
It is embedded in autonomous driving network using sensing module pre-training result as initial value herein, and increases the more of avoidance planning Tasking learning module also increases for planning layer network so that avoidance feature serves not only as the input feature vector of control module and explains, adjusts Excellent approach.
Increased obstacle avoidance module is as multi-task learning as a result, playing the direct feature structure to control module herein D is describediWith control result aiPrevention Intervention.And establish that loss function is as shown in Equation 15, and wherein m, l, r respectively represent center, a left side The distance parameter of side, the right side visual field makes in loss function since obstacle avoidance module is to the higher dependence of central field of vision, increase pair Central field of vision miPunishment dynamics, and to left li, right riVisual field penalty term passes through μlWith μrControl, finally in an experiment punishes the two Penalize dynamics is same to be set as 0.5.
Generally speaking for autonomous driving network, select ReLU as the activation primitive after all hidden layers herein, and herein On the basis of, increase BN before activation primitive and normalize layer, to solve situations such as data distribution is uneven.Using 20% and 50% Dropout is respectively applied to convolutional layer and full articulamentum, so as to improve the generalization ability of model.Definition herein acts aiFor ai= <si,ti,bi>, correspond respectively to wheel steering si, throttle acceleration ti, brake feedback bi.Due to finding vehicle in an experiment Throttle is not identical as the sensitivity of brake feedback, and for example throttle does not have very the feedback of 0-0.3 on driving experience Big gap, but brake for the feedback of 0-0.3 is but enough that the vehicle just travelled at crossing is allowed effectively to be braked, if only pressing It seeks unity of standard and determines loss function, then the movement and practice movement for being easy to allow network to be mistakenly considered prediction are very close to but practical former The scene that this needs is braked but accidentally is fed back into acceleration by network.Then loss function is defined herein as shown in formula 16-17, wherein gt Represent ground truth.
In network training, set epoch as 100, batch size be 120, and use Adam[35]Optimizer joins network Number optimizes, and wherein beta1 is set as 0.7, beta2 and is set as 0.85, and initial learning rate is set as 0.0002.Whenever network After having carried out the training of 50000 steps, the learning rate for the half that can decay, to prevent loss value after excessive training from generating local oscillation, net The problems such as network can not restrain.
6. experimental result and analysis
This chapter experiment will be built emulation respectively with simulation parameters such as different weather conditions, different road concentrations and be surveyed Test ring border, and the autonomous driving network of building is tested, using interference feedback success rate and average accurate driving distance (nothing It is collision, violating the regulations) experimental result is judged as the reference standard of network autonomous driving ability, and with current other are end-to-end Autonomous driving network is compared and analyzes, the final advantage for summarizing this paper autonomous driving network and deficiency.
Herein in an experiment, it is trained and verifies using the city that Carla is carried, wherein using Town1 as main instruction Practice scene, using Town2 as main verifying scene, by autonomous driving network adapter tube vehicle control system, to realize to vehicle Autonomous driving.32 pairs are chosen herein to test at a distance of 1 kilometer or more of RoadNode as verifying example.
In experimental test procedures, autonomous driving network show the study to difficult task (avoidance, planning, steering) at Effect, but be the autonomous driving ability of comprehensive test network, herein according to bad weather degree and the road degree of crowding to experiment number According to being divided.Wherein weather conditions are classified by normal, complicated, severe three kinds of situations, rainy day, night are defined as multiple Miscellaneous weather conditions, by multiple complicated weather conditions and the weather deposited is defined as bad weather, remaining is normal weather.Road is crowded Degree is then divided into spacious, smooth and congestion three classes by vehicle/pedestrian's quantity, and using quantity 10,20 as point of three classes Boundary mark is quasi-.
The autonomous driving effect for being network under different weather scene, it is seen that with the increase of scene complexity, scene Validity feature becomes more to obscure, and the sensing capability to network is also bigger challenge.It is herein that experimental data is vaporous according to day Condition is divided, and is summarized to cities and towns 1 and the interference feedback situation under different weather situation in cities and towns 2, is shown in Table 2-3.By table In as it can be seen that the feedback success rate in test scene is generally lower than Training scene, and heterogeneous networks model has in normal weather Higher interference feedback effect, and heterogeneous networks model is owned by the ability to different weather situation rational learning.But phase For other network models, this paper autonomous driving network is all demonstrated by higher interference feedback ability under different weather situation.
Interference feedback success rate is driven under 2 cities and towns of table, 1 different weather situation
Interference feedback success rate is driven under 3 cities and towns of table, 2 different weather situation
Average accurate driving distance (km) under 4 cities and towns of table, 1 different weather situation
Average accurate driving distance (km) under 5 cities and towns of table, 2 different weather situation
According to the above analysis, the average accurate driving distance under cities and towns 1 and 2 different weather situation of cities and towns is carried out respectively total Knot, is shown in Table shown in 4-5.Although showing each network sensing capability certain to target scene according to interference feedback ability, The accuracy and stability of each automatic network autonomous driving ability are embodied by the way that driving distance is more intuitive.This paper network is also shown The ability of the autonomous driving ability more stable relative to comparison network and more acurrate reply disturbance regime under different weather situation.
Interference feedback success rate is driven under the different congestion density in 6 cities and towns of table 1
Interference feedback success rate is driven under the different congestion density in 7 cities and towns of table 2
It is anti-to cities and towns 1 and the interference under different roading densities in cities and towns 2 respectively herein for different roading density situations Feedback situation is summarized, and is shown in Table shown in 6-7.As can be seen from the table, more network shows relatively good interference under spacious scene Feedback capability, but document[3]With document[4]But in the increased situation of congestion density, weaker interference feedback energy is shown Power.And relatively higher interference feedback ability is then all maintained under different roading densities herein.
According to the above analysis, the average accurate driving distance under the congestion density different from cities and towns 2 of cities and towns 1 is carried out respectively total Knot, is shown in Table shown in 8-9.As can be seen from the table, although document[3]With document[4]In spacious environment, do not reach higher anti-dry Effect, but accurate driving distance with higher are disturbed, this generates larger contrast with interference feedback result.By to data It is repeatedly found after analysis, main reason is that comparison network is not high for the accurate feedback ability for colliding interference noise, still In spacious environment, even if network does not timely respond to interference noise and can also maintain correctly to drive posture.However it if is directed to More crucial effect is then played in the stability of the higher scene of density, the planning avoidance of each network.It is close to be directed to different congestions Scene is spent, this paper network all keeps accurate driving distance relatively far away from, embodies the more stable autonomous driving ability of network.
Average accurate driving distance (km) under the different congestion density in 8 cities and towns of table 1
Average accurate driving distance (km) under the different congestion density in 9 cities and towns of table 2
Each network of table 10 drives interference feedback success rate
Each network of table 11 is averaged accurate driving distance (km)
By above data by interference feedback success rate and average experiment knot of the driving distance to each network under different cities and towns Fruit is summarized, and is shown in Table shown in 10-11.By being analyzed above with data in table as it can be seen that increasing collision and multiple dimensioned sensing module After study, network has interference noise apparent feedback effects, compared to document[4]30% feedback rates, this paper network tool There is 73% higher feedback result, obstacle avoidance module intervention is embodied with this and for the importance of inquiry learning.For increased FPN The pre-training feature extraction network of+PPM, can from higher noise maintained under each weather conditions feedback and driving distance result Find out it compared to document[3]With document[4]More stable, comprehensive sensing capability.Also just because of the elder generation of this network structure Sensing capability is tested, network is just made to be understood that the advanced driving behavior such as avoidance, planning, and is carried out more for effective study.

Claims (1)

1. network structureization, is layered first to improve network by the autonomous driving network under a kind of multiple dimensioned perception and Global motion planning The interpretability of each section secondly by pre-training FPN network [15] to improve the multiple dimensioned sensing capability of network, and increases and keeps away Barrier module is to optimize vehicle planning ability, eventually by perception avoidance feature and planning navigation instruction control control flow direction, with reality Existing vehicle is intended to the overall situation and the effective Feedback of local feature, it is characterized in that:
Multiple dimensioned sensing module based on FPN
The end-to-end autonomous driving network characterization extraction stage obtains the high semantic feature of top layer by successively feedforward convolution algorithm at present, The characteristic extraction procedure realized even so has certain sensing capability to high semantic feature, but the feature carried by it Scale size is limited, so currently advanced autonomous driving network is to Analysis On Multi-scale Features and does not have good perceived effect, and And due to the end-to-end learning by imitation mode of current autonomous driving network, structured objects (vehicle, pedestrian etc.) can not be carried out It is effective to distinguish, cause to focus mainly on road boundary in network characterization extraction process and the structuring mesh of ignoring no less important Mark, the similarly design due to network black box itself, make current autonomous driving network lack layering tuning channel, i.e., if There is exception control as a result, can not determine it is which, there are problems in stage in network, can only increase strengthening training simply, not Preferable prioritization scheme, and herein then in view of the above problems, use for reference currently advanced multiscale target detection method, and combine mesh Detection zone regression scheme, the final perception for realizing network to Multi-scale model target are marked, and provides region for sensing network Explanatory channel will be amplified to multiscale target detection method from conventional target detection method herein, and be built on this basis Sensing network explains network eventually by provincial characteristics, increases sensing network region interpretation approach, completes to multiple dimensioned perception net The whole design of network；
Traditional CNN (Convolutional NeuralNetwork) target detection feature extraction process, such as Faster RCNN[11], only depend on and top-level feature predicted, although high-level characteristic possesses richer semantic information, feature at this time Position Relative Fuzzy and the particularity due to upper sampling process, cause many small scale features micro- in top-level feature figure Its is micro-, even if similar SSD network[14]It to solve the problems, such as multiple dimensioned extraction, is predicted, is solved using the method for different characteristic figure Determined part multiple scale detecting the problem of, but still have ignored the feature of the small scale of many bottoms；As shown in Fig. 1 (a), tradition Feature prediction only consider a semantic feature of top layer P3, but more fully information many for P1, P2 layers is but ignored, and causes Multiple dimensioned feature can not be effectively extracted, if it is assumed that, P1, P2, P3 are adopted and combined by some way, both Again include multiple dimensioned rudimentary semantic feature comprising high-level semantics feature, can effectively solve the problems, such as this, and FPN (Feature Pyramid Networks) network[15]Proposition be namely based on such a it is assumed that as shown in Fig. 1 (b), CNN network is no longer located In linear structure, and become the feature pyramid structure of top-down lateral connection, every layer of characteristic pattern all with same scale feature figure Association, each scale feature figure can all predict different characteristic, reach the effective extraction to Analysis On Multi-scale Features and utilize, FPN network Comprehensive concern to low-level image feature is realized by the fusion to same scale characteristic pattern, it is made to reach good to multiscale target Recognition effect；
FPN network as shown in formula 1-2, whereinFor n-th of characteristic pattern after fused, g is FPN structure, and v then inserts for bilinearity Value function, fm×mThen correspond to the convolution operation carried out from the convolution kernel of different m × m sizes, φnThen character pair extracts network n-th Layer characteristic pattern, I is original image；The FPN network so realized, although can have preferable recognition capability to multiple dimensioned, It is that information transfer capacity is weaker between characteristic layer non-conterminous in top-down fusion process, in the fusion Jing Guo multilayer Afterwards, the influence of farther away characteristic layer adjacent with current layer but becomes very little, so the different rulers obtained herein in FPN network After the characteristic pattern of degree, by the way of unified scale feature fusion, so that characteristic pattern is not solely dependent on single scale, also make to feel Know that layer has unified receptive field, the perception information to planning layer output redundancy is reduced, such as the sensing network that Fig. 2 is built herein Shown in Fused module；In sensing network is built, herein in conjunction with FPN and PSPnet[16], and in feature pyramid network Top layer is added to PPM (Pyramid Pooling Module) pyramid pond module, and PPM is the most crucial network knot of PSPnet Structure mainly passes through and increases CNN overall situation receptive field, to achieve the purpose that multiple dimensioned pond, is finally obtained based on above-mentioned theory herein Obtain multiple dimensioned perception fusion feature figure P10；
φn=fn(φn-1)=fn(fn-1(...f1(I))) (2)
Herein on the basis of combining multiple dimensioned perception information, to obtain more structured objects features as far as possible, RPN is chosen (Region Proposal Network) region candidate network[11]The perceptually interpreter of module carries out structured objects Candidate region extraction is carried out, so that sensing layer has structured objects recognition capability and network interpretation ability, RPN network is substantially The function that selective search i.e. region is suggested is realized, RPN network obtains each by carrying out sliding window operation to characteristic pattern The area score of sliding window position and position return revised region and suggest, finally carry out non-maxima suppression to candidate region, To obtain the testing result after perception interpretation layer understands feature；
As shown in figure 3, joint-detection regression block constructs the judge network to sensing results herein on the basis of sensing network, The channel of detection and tuning is provided for sensing network, and network will be judged as target detection network and carry out structuring perception target Pre-training realizes that sensing network to the priori understandability of structured features, and after being implanted into end-to-end autonomous driving model, is incited somebody to action Perception pre-training parameter is trained as initial parameter, will test recurrence layer, perceptually the interpreter of layer, is deepened pair with this The understandability of sensing network；
Herein in conjunction with FPN, PPM and RPN network, weak multiple dimensioned sensing capability is mentioned with poor sensing network interpretability Corresponding solution out, and then design autonomous driving sensing network；The sensing network designed even so, increases overall network The workload of pre-training, but be the study of network autonomous driving in the future, optimization tuning etc., open more succinct intuitive sense Know shortcut, it is believed that in the course of end-to-end autonomous driving network fast development, this pretreated method of perception can become certainly The universal pre-training mode of main driving network；
Based on the planning module apart from avoidance
Avoidance problem is always to be difficult the ability of autonomous learning in learning by imitation, but due to the irreplaceability of module itself, make It obtains it and is increasingly becoming one of the important indicator for measuring autonomous driving effect superiority and inferiority, although presently, there are the sides of related avoidance Method[25-27], but it is not provided with the avoidance scheme of depth learning by imitation, accomplish effectively so that nowadays autonomous driving network is more difficult Avoidance, and very big reason is excessive learning parameter, non-structured learning characteristic and without preferably study side among these Caused by method；
The mode of new characteristic present method and study is proposed herein in conjunction with the advantage of learning by imitation, and as control module Priori features carry out control forecasting, traditional avoidance problem, as shown in Equation 3, wherein c be drive vehicle, d be vehicle safety away from From b is barrier, and O is object boundary point set, and i is the quantity of t moment barrier；
D < Ot(bi)-Ot(c) (3)
Although this avoidance mode can ensure the safe driving at each moment, but not consider influence of the speed to braking distance And many factors such as driving experience, it is only applicable to the non-vehicle for multiplying fortune and driving of low speed；
Using speed v as intelligent barrier avoiding reference parameter on the basis of herein, formula 4 can be obtained；
D < Ot(bi)-Ot(c)+(vt(bi)-vt(c))t (4)
Wherein v is that t moment corresponds to the speed of object, if only considering non-retrograde situation, i.e. v at this timet(bi)≤0, only it need to meet formula 5, Safe avoidance under fast state can be realized, avoidance mode is also made to become relying on itself shifting speed and observe reflecting for moment relative distance Function is penetrated, since sensing network only extracts the feature at forward sight visual angle, so obstacle avoidance module is increased speed on this basis and distance Mapping relations, and result is directly acted on into control module；
D < Ot(bi)-Ot(c)+vt(c)t (5)
Herein based on above-mentioned analysis, design avoidance network is as shown in Equation 6, whereinFor apart from fusion feature,For perception fusion Feature, s are current vehicle speed, and fc is full connection operation, and m is full connection dimension, and increases for avoidance planning network and judge module, is expanded Exhibition formula 6 obtains formula 7, and wherein d is that avoidance returns distance, and the accuracy of avoidance web results is judged with this；
Based on view of the above, avoidance network as shown in Figure 4 is built herein, according to avoidance mapping relations, to guarantee distance feature Integrality introduces velocity characteristic and the two is merged full connection result merges with distance feature, make result utmostly according to Rely under the premise of the two feature, do not destroy prior distance feature, fusion results can be passed respectively as distance feature Enter subsequent control module, and obtains and judge distance results；
The advantage being designed in this way is that priori velocity information adjusts the distance prediction and avoidance obstacle all has great importance, and is melting Image distance feature is not destroyed on the basis of conjunction speed, under the premise of guaranteeing obstacle avoidance module to the dependence of the two, makes network training More standby stressing property；
Control module based on Global motion planning
Since end-to-end autonomous driving network learns by imitation mankind's driving data, cause the driver behavior finally learnt complete Dependent on the Driving Scene of acquisition, so when there is alternative path in Driving Scene (such as crossing), the independent nothing from image Method determines the real driving intention of driver；It then needs to introduce Global motion planning scheme at this time, in internal system according to Global motion planning knot Fruit provides planning path, makes autonomous driving network in alternative path, obtains unique planning path, final to solve independently to drive It sails network and makes the control result for violating system intention in more options path[28]；
Document is used for reference herein[4]Control module screening technique, design this paper Global motion planning control network, first calculating current location Global motion planning instruction, each corresponding difference of planning instruction are provided and to current driving behavior with target position optimum programming path Branch is controlled, realizes that network can take correctly control feedback under different demands for control；
As shown in figure 5, conditional order c as Global motion planning instruct, switching control module branch network, shared upper layer perception and Under the premise of avoidance information, consistency and uniformity that driving intention flows to control module, herein, this control are realized Molding block particularly may be divided into along follow, intentions of turning left, turn right be intended to, keep straight on intention etc.；
End-to-end autonomous driving core network
It herein will be in learning by imitation theoretical basis[24]In conjunction with above-mentioned branching networks, finally to whole autonomous driving core network into Row master-plan and analysis；
Autonomous driving learning by imitation, which is laid particular emphasis on, learns the subjective driving condition of professional driver, imitates and drives under each scene The driving behavior and driving purpose for the person of sailing, finally realize learning by imitation[29-32]；Set herein one in discrete time node t and With the controller M of environmental interaction, current observed information o is received in each timing node t, controller MtAnd it takes corresponding anti- Feedback acts at, that is, the process of feedback of a controller M is completed, and the core concept of learning by imitation is the driving behavior according to the mankind Such a controller M of training makes its feedback action more approach the movement of the mankind, then defines training set D, as shown in Equation 8, Wherein by the currently active observation oiAnd a is fed back in the currently active movementiPrimary effective training sample is constituted, it is assumed that human action Accurate feedback action successfully is being taken to field of view every time, that need to define one can approach the mankind as far as possible The mapping function of feedback behavior can realize basic learning by imitation, as shown in Equation 9
The mapping function F for wherein defining an imitation controlling behavior, passes through current time oiWith corresponding parameter θ, the corresponding moment is obtained Ai, when mapping function F most approaches mankind's feedback action, can realize the training of learning by imitation；
For perceiving branching networks, sensing results mapping function P is defined, current time o is passed throughi, and corresponding parameter ε, obtain perception Area detection result pi, as shown in Equation 10, in perception branching networks pre-training, only the parameter ε of function P need to be made to be substantially equal to piAnnotation results, that is, complete perception branching networks pre-training；
For avoidance planning network, avoidance planning is defined apart from mapping function B, passes through current time Perception Features vector pfiWith speed Spend vector siAnd corresponding parameter lambda, obtain avoidance distance di, as shown in Equation 11, and since learning by imitation mapping relations are more difficult Realize the ability of avoidance, i.e. vehicle can not be according to sensing results oiDue avoidance feedback is made, so herein by velocity vector si As the output feature that another input is perceived with perception branch outcome collectively as avoidance, as shown in Equation 12；
Network is controlled for Global motion planning, the output result of driver no longer only dominated by observation result, is relied more on and is State inside system, the vector h of definition characterization internal system state, it is acted as with observation result o, velocity vector s mono- at this time For the reference of feedback action, i.e. ai=E (oi,si,hi), to solve the barrier that the instruction of system inside and outside is linked up, wherein h includes and makes The subjective intention of user, the state of the internal systems such as system Global motion planning instruction input c=c eventually by additional command is introduced (h), system sneak condition is made to become the important references factor of control instruction, as shown in Equation 13.It is designed in conjunction with overall network, finally Learning training diversity 8 will initially be imitated and extend an accepted way of doing sth 14；
The end-to-end automatic Pilot network designed herein, is made of, as shown in Figure 6 sensing layer, planning layer, control layer；Sensing layer By input Image i, FPN f and Fused module composition.Wherein Image i is to receive 300 be converted into from camera sensing device × 100 RGB image, FPN module then increase PPM structure on the basis of FPN, structure foundation PSPnet network design theory, Design PPM is that global average pond layer and 3 × 3 convolutional layers, Fused fused layer are merged by the way of bilinear interpolation, Perception characteristic results are obtained eventually by the full articulamentum of 512 dimensions, realize effective extraction of the sensing layer to Analysis On Multi-scale Features, perception Interpretation layer does not embody in core network, but in training and optimization process, needs to perceive fusion feature figure and is passed to RPN net In network, obtain region candidate as a result, sensing network is explained and optimized with this, planning layer then respectively by obstacle avoidance module, melt Molding block, planning instruction valve are constituted, and the ring type that wherein obstacle avoidance module constructs 3 layer of 128 neuron according to avoidance mapping relations is complete Obstacle avoidance module is connected, the input using avoidance characteristic results as Fusion Module and the input for returning distance d, to avoidance result It is iterated, fused layer merges perception and avoidance feature and uniformly consigns to planning instruction valve c, selects accordingly control network, controls Molding block is made of four control branching networks, and each control branching networks are made of two full connections and a recurrence movement a, Control layer is instructed by Global motion planning, is selected corresponding control module, is established the communications conduit of outer bound pair network, realizes Global motion planning Under learning by imitation.
CN201910607644.XA 2019-07-08 2019-07-08 It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network Pending CN110427827A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910607644.XA CN110427827A (en) 2019-07-08 2019-07-08 It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910607644.XA CN110427827A (en) 2019-07-08 2019-07-08 It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network

Publications (1)

Publication Number Publication Date
CN110427827A true CN110427827A (en) 2019-11-08

Family

ID=68410327

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910607644.XA Pending CN110427827A (en) 2019-07-08 2019-07-08 It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network

Country Status (1)

Country Link
CN (1) CN110427827A (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111105031A (en) * 2019-11-11 2020-05-05 北京地平线机器人技术研发有限公司 Network structure searching method and device, storage medium and electronic equipment
CN111401517A (en) * 2020-02-21 2020-07-10 华为技术有限公司 Method and device for searching perception network structure
CN111538328A (en) * 2020-04-03 2020-08-14 浙江工业大学 Priority hierarchical prediction control method for obstacle avoidance trajectory planning and tracking control of autonomous driving vehicle
CN111860153A (en) * 2020-01-09 2020-10-30 九江学院 Scale-adaptive hyperspectral image classification method and system
CN111931790A (en) * 2020-08-10 2020-11-13 武汉慧通智云信息技术有限公司 Laser point cloud extraction method and device
CN111975775A (en) * 2020-08-13 2020-11-24 山东大学 Autonomous robot navigation method and system based on multi-angle visual perception
CN112232490A (en) * 2020-10-26 2021-01-15 大连大学 Deep simulation reinforcement learning driving strategy training method based on vision
CN112380923A (en) * 2020-10-26 2021-02-19 天津大学 Intelligent autonomous visual navigation and target detection method based on multiple tasks
CN112541408A (en) * 2020-11-30 2021-03-23 北京深睿博联科技有限责任公司 Feasible region identification method, device, equipment and computer readable storage medium
CN112859810A (en) * 2021-01-13 2021-05-28 自行科技(武汉)有限公司 ADAS algorithm verification method and device based on Carla platform
CN112926370A (en) * 2019-12-06 2021-06-08 纳恩博（北京）科技有限公司 Method and device for determining perception parameters, storage medium and electronic device
CN113009453A (en) * 2020-03-20 2021-06-22 青岛慧拓智能机器有限公司 Mine road edge detection and map building method and device
CN113029151A (en) * 2021-03-15 2021-06-25 齐鲁工业大学 Intelligent vehicle path planning method
CN113095241A (en) * 2021-04-16 2021-07-09 武汉理工大学 Target detection method based on CARLA simulator
CN113361643A (en) * 2021-07-02 2021-09-07 人民中科(济南)智能技术有限公司 Deep learning-based universal mark identification method, system, equipment and storage medium
CN114332590A (en) * 2022-03-08 2022-04-12 北京百度网讯科技有限公司 Joint perception model training method, joint perception device, joint perception equipment and medium
US20220232357A1 (en) * 2021-01-15 2022-07-21 Harman International Industries, Incorporated V2x communication system with autonomous driving information
CN116626670A (en) * 2023-07-18 2023-08-22 小米汽车科技有限公司 Automatic driving model generation method and device, vehicle and storage medium

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111105031A (en) * 2019-11-11 2020-05-05 北京地平线机器人技术研发有限公司 Network structure searching method and device, storage medium and electronic equipment
CN111105031B (en) * 2019-11-11 2023-10-17 北京地平线机器人技术研发有限公司 Network structure searching method and device, storage medium and electronic equipment
CN112926370A (en) * 2019-12-06 2021-06-08 纳恩博（北京）科技有限公司 Method and device for determining perception parameters, storage medium and electronic device
CN111860153B (en) * 2020-01-09 2023-10-13 九江学院 Scale-adaptive hyperspectral image classification method and system
CN111860153A (en) * 2020-01-09 2020-10-30 九江学院 Scale-adaptive hyperspectral image classification method and system
CN111401517B (en) * 2020-02-21 2023-11-03 华为技术有限公司 Method and device for searching perceived network structure
CN111401517A (en) * 2020-02-21 2020-07-10 华为技术有限公司 Method and device for searching perception network structure
CN113009453A (en) * 2020-03-20 2021-06-22 青岛慧拓智能机器有限公司 Mine road edge detection and map building method and device
CN113009453B (en) * 2020-03-20 2022-11-08 青岛慧拓智能机器有限公司 Mine road edge detection and mapping method and device
CN111538328A (en) * 2020-04-03 2020-08-14 浙江工业大学 Priority hierarchical prediction control method for obstacle avoidance trajectory planning and tracking control of autonomous driving vehicle
CN111538328B (en) * 2020-04-03 2022-07-26 浙江工业大学 Priority hierarchical prediction control method for obstacle avoidance trajectory planning and tracking control of autonomous driving vehicle
CN111931790A (en) * 2020-08-10 2020-11-13 武汉慧通智云信息技术有限公司 Laser point cloud extraction method and device
CN111975775A (en) * 2020-08-13 2020-11-24 山东大学 Autonomous robot navigation method and system based on multi-angle visual perception
CN111975775B (en) * 2020-08-13 2022-05-27 山东大学 Autonomous robot navigation method and system based on multi-angle visual perception
CN112380923A (en) * 2020-10-26 2021-02-19 天津大学 Intelligent autonomous visual navigation and target detection method based on multiple tasks
CN112232490B (en) * 2020-10-26 2023-06-20 大连大学 Visual-based depth simulation reinforcement learning driving strategy training method
CN112232490A (en) * 2020-10-26 2021-01-15 大连大学 Deep simulation reinforcement learning driving strategy training method based on vision
CN112541408A (en) * 2020-11-30 2021-03-23 北京深睿博联科技有限责任公司 Feasible region identification method, device, equipment and computer readable storage medium
CN112541408B (en) * 2020-11-30 2022-02-25 北京深睿博联科技有限责任公司 Feasible region identification method, device, equipment and computer readable storage medium
CN112859810A (en) * 2021-01-13 2021-05-28 自行科技(武汉)有限公司 ADAS algorithm verification method and device based on Carla platform
US20220232357A1 (en) * 2021-01-15 2022-07-21 Harman International Industries, Incorporated V2x communication system with autonomous driving information
US11463851B2 (en) * 2021-01-15 2022-10-04 Harman International Industries, Incorporated V2X communication system with autonomous driving information
CN113029151A (en) * 2021-03-15 2021-06-25 齐鲁工业大学 Intelligent vehicle path planning method
CN113095241A (en) * 2021-04-16 2021-07-09 武汉理工大学 Target detection method based on CARLA simulator
CN113361643A (en) * 2021-07-02 2021-09-07 人民中科(济南)智能技术有限公司 Deep learning-based universal mark identification method, system, equipment and storage medium
CN114332590A (en) * 2022-03-08 2022-04-12 北京百度网讯科技有限公司 Joint perception model training method, joint perception device, joint perception equipment and medium
CN116626670A (en) * 2023-07-18 2023-08-22 小米汽车科技有限公司 Automatic driving model generation method and device, vehicle and storage medium
CN116626670B (en) * 2023-07-18 2023-11-03 小米汽车科技有限公司 Automatic driving model generation method and device, vehicle and storage medium

Similar Documents

Publication Publication Date Title
CN110427827A (en) It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network
Ma et al. Artificial intelligence applications in the development of autonomous vehicles: A survey
Van Brummelen et al. Autonomous vehicle perception: The technology of today and tomorrow
US20220171390A1 (en) Discrete Decision Architecture for Motion Planning System of an Autonomous Vehicle
Elallid et al. A comprehensive survey on the application of deep and reinforcement learning approaches in autonomous driving
CN110796856B (en) Vehicle lane change intention prediction method and training method of lane change intention prediction network
CN106874597B (en) highway overtaking behavior decision method applied to automatic driving vehicle
CN111367282B (en) Robot navigation method and system based on multimode perception and reinforcement learning
CN110531753A (en) Control system, control method and the controller of autonomous vehicle
CN110562258B (en) Method for vehicle automatic lane change decision, vehicle-mounted equipment and storage medium
CN110531754A (en) Control system, control method and the controller of autonomous vehicle
CN1915725B (en) Driver assistance system
CN108319249A (en) Unmanned algorithm synthesis evaluation system based on driving simulator and method
CN107886750B (en) Unmanned automobile control method and system based on beyond-visual-range cooperative cognition
CN106198049A (en) Real vehicles is at ring test system and method
CN107009968A (en) Mobile lidar control method, device and mobile unit
Hafeez et al. Insights and strategies for an autonomous vehicle with a sensor fusion innovation: A fictional outlook
Zhang et al. A cognitively inspired system architecture for the Mengshi cognitive vehicle
CN112258841A (en) Intelligent vehicle risk assessment method based on vehicle track prediction
CN115662166B (en) Automatic driving data processing method and automatic driving traffic system
Zöldy et al. Cognitive mobility–cogmob
JP2021043622A (en) Recognition model distribution system and recognition model updating method
Tippannavar et al. SDR–Self Driving Car Implemented using Reinforcement Learning & Behavioural Cloning
Raipuria et al. Road infrastructure indicators for trajectory prediction
CN111508256A (en) Traffic information reconstruction method based on regional time-space domain and intelligent traffic system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20191108

WD01 Invention patent application deemed withdrawn after publication