CN115285143A - Automatic driving vehicle navigation method based on scene classification - Google Patents
Automatic driving vehicle navigation method based on scene classification Download PDFInfo
- Publication number
- CN115285143A CN115285143A CN202210925922.8A CN202210925922A CN115285143A CN 115285143 A CN115285143 A CN 115285143A CN 202210925922 A CN202210925922 A CN 202210925922A CN 115285143 A CN115285143 A CN 115285143A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- scene
- scene classification
- model
- navigation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000003044 adaptive effect Effects 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims abstract description 6
- 238000013145 classification model Methods 0.000 claims description 56
- 238000012549 training Methods 0.000 claims description 37
- 239000011159 matrix material Substances 0.000 claims description 32
- 238000005259 measurement Methods 0.000 claims description 27
- 238000001914 filtration Methods 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 24
- 238000005516 engineering process Methods 0.000 claims description 14
- 238000013527 convolutional neural network Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 238000009826 distribution Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 4
- 230000008447 perception Effects 0.000 claims description 4
- 239000004576 sand Substances 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000001133 acceleration Effects 0.000 claims description 3
- 210000002569 neuron Anatomy 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 2
- 230000007613 environmental effect Effects 0.000 abstract description 7
- 238000013507 mapping Methods 0.000 abstract description 3
- 238000004458 analytical method Methods 0.000 abstract description 2
- 241000282414 Homo sapiens Species 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- KJLLKLRVCJAFRY-UHFFFAOYSA-N mebutizide Chemical compound ClC1=C(S(N)(=O)=O)C=C2S(=O)(=O)NC(C(C)C(C)CC)NC2=C1 KJLLKLRVCJAFRY-UHFFFAOYSA-N 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 239000011664 nicotinic acid Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0002—Automatic control, details of type of controller or control system architecture
- B60W2050/0004—In digital systems, e.g. discrete-time systems involving sampling
- B60W2050/0005—Processor details or data handling, e.g. memory registers or chip architecture
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Navigation (AREA)
Abstract
The invention discloses an automatic driving vehicle navigation method based on scene classification, relating to the technical field of automatic driving vehicle navigation; the navigation capability of the automatic driving vehicle is improved when resources are limited, particularly in complex urban scenes, classification of scene trafficability is introduced, the navigation direction is determined through rough semantic analysis of environmental information, the vehicle is controlled to travel, the accurate modeling of the environment and the recognition process of objects in images are avoided, the unmanned vehicle is guided to run by utilizing direct mapping between the scenes and actions, and therefore a navigation strategy adaptive to consumption level is achieved.
Description
Technical Field
The invention relates to the technical field of automatic driving vehicle navigation, in particular to an automatic driving vehicle navigation method based on scene classification.
Background
The navigation technology of the automatic driving vehicle depends on the cooperative cooperation of computer vision, radar, a monitoring device, a global positioning system and the like, so that the vehicle can move from one position to another position without the active operation of human beings. Because the automatic driving navigation technology does not need human driving, the driving error of human can be effectively avoided theoretically, the occurrence of traffic accidents is reduced, and the transportation efficiency of roads can be improved. As a result, autonomous vehicle navigation technology is gaining increasing attention.
The traditional navigation technology mainly depends on a global navigation satellite system (such as GPS), but because the traditional navigation technology is easy to be shielded and has multipath effect, the high-precision positioning and navigation are difficult to realize in complex scenes such as urban canyons, jungles and the like. In recent years, many satellite-independent navigation technologies such as inertial navigation, visual navigation, database matching navigation, bionic navigation, collaborative navigation and the like are researched and developed at home and abroad. The typical example of the method is a "map" based navigation technology, that is, a high-performance sensor (such as a laser radar) and a map (such as a high-precision map) are used for feature matching to realize accurate navigation. However, in some highly dynamic and complex scenarios, it is difficult to achieve an exact match to the environment, especially when global information is lost, it becomes strange, which is not unlike GPS outage. In addition, the expensive sensor also restricts the vector production, and is difficult to adapt to the consumption level requirement.
In order to meet the consumer-level navigation requirements, how to avoid the dependence on a high-performance sensor and realize a navigation task by using a low-cost visual sensor is a technical problem to be solved.
At present, artificial intelligence grows up rapidly, the machine simulation animal vision technology is mature gradually, a new opportunity is provided for high-precision guidance, and low-cost vision navigation is expected to become a wind vane of the next generation unmanned system navigation technology. Therefore, based on the advanced artificial intelligence technology, the novel low-cost navigation method and system can provide a new solution for vehicle navigation and provide important technical support for high-precision navigation in mass production.
Disclosure of Invention
The invention aims to provide an automatic driving vehicle navigation method based on scene classification, which improves the navigation capability of an automatic driving vehicle when resources are limited, particularly introduces scene trafficability classification in complex urban scenes, determines the navigation direction through rough semantic analysis of environmental information, controls the vehicle to advance, bypasses the precise modeling of the environment and the precise identification process of objects in images, and guides the operation of an unmanned vehicle by utilizing direct mapping between the scene and actions, thereby realizing a navigation strategy adaptive to consumption level.
In order to achieve the purpose, the invention adopts the technical scheme that:
an automatic driving vehicle navigation method based on scene classification comprises the following steps:
s1: acquiring and classifying images of the surrounding environment of the automatic driving vehicle by using a vehicle-mounted camera; establishing a scene classification model based on a convolutional neural network technology, and training the scene classification model;
s2: based on an extended Kalman filtering framework, a system state model is established, the prior estimation of the state variables of the step lines is predicted by using extended Kalman filtering, a covariance matrix of the prior state is updated in real time by using an adaptive algorithm based on filtering information, and the posterior estimation of the state variables of the step lines is updated by using extended Kalman filtering, so that the adaptive positioning of the vehicle is realized;
s3: selecting a vehicle control mode according to the classification result of the scene classification model to control the running of the vehicle;
s4: establishing an automatic driving automobile navigation frame; the navigation process of the navigation framework comprises the following steps: and collecting the surrounding environment image of the automatic driving vehicle at the current moment as the input of a scene classification model, carrying out the driving control of the vehicle according to the output of the scene classification model, and then carrying out the position judgment of the current vehicle according to a self-adaptive positioning result, and repeating the steps until the automatic driving vehicle reaches the target position.
The S1 specifically comprises the following steps:
s1.1: acquiring a sample data set: shooting scene pictures of different environments by using a vehicle-mounted camera, and classifying the scene pictures;
s1.2: establishing a scene classification model based on a convolutional neural network architecture, carrying out normalization processing on a data set, and training the established scene classification model by using a training set;
s1.3: adjusting the hyper-parameters of the scene classification model according to the number of the object types in the scene, wherein the hyper-parameters comprise a learning rate, an attenuation rate, iteration times, the number of layers, the number of neurons in each layer, batch size and the weight of each part in a loss function; and repeating S1.2, and training the scene classification models suitable for different complexities, including high, medium and low complexity scene classification models.
The difference of the scene complexity represents the difference of the number of things contained in the scene picture in the sample data set; the complexity is low when the number of the object types is less than or equal to three, medium complexity when the number of the object types is more than three and less than seven, and high complexity when the number of the object types is more than seven.
The S1.1 specifically comprises the following steps:
s1.1.1: manually controlling the vehicle to run, acquiring an environment image through a camera arranged at a preset position of the vehicle, generating a sample data set, and recording the shooting direction of each sample data;
the camera can be twisted by plus or minus eighty degrees; the sample data shooting direction is divided into five directions, namely a front direction, a front left direction, a front right direction, a left direction and a right direction;
s1.1.2: performing semantic classification on each sample scene picture in the sample data set, and selecting a scene classification label for each sample scene picture according to an actual scene;
the semantic classification represents classifying things in the sample scene picture according to categories, including ten categories of highways, non-motor vehicle roads, pedestrians, vehicles, buildings, street lamps, trees, grasslands, sand lands and sky; the scene classification labels comprise fifteen label categories, namely three conditions of priority, feasibility and infeasibility respectively corresponding to the right front, the left front, the right front, the left side and the right side;
s1.1.3: and dividing the sample data set subjected to scene classification into a training set and a test set according to a proportion.
The S1.2 specifically comprises the following steps:
s1.2.1: establishing a scene classification model based on a shallow convolutional neural network architecture based on a Tensorflow open source framework; the model comprises a convolution layer, a pooling layer and a full-connection layer; outputting x characteristic graphs by each convolution and pooling layer, performing characteristic fusion by a full connection layer, and dividing a scene into fifteen probability values by utilizing a Softmax function to output; as shown in formula (1):
wherein z is i Outputting a value for the ith node of the full connection layer; z is a radical of c Outputting a value for the c-th node of the full connection layer;
s1.2.2: adjusting the training set sample subjected to scene classification processing into y x y size as the input of a scene classification model, and performing unified normalization processing on the training set and the test set:
x norm ＝(x RGB -min)/(max-min)-b (2)
wherein x is RGB Color values for three channels; max and min are the extreme values of the current image,set to 255 and 0, respectively; adjust the pixel intensity of each image to [ b, b]；
S1.2.3: training a scene classification model by using a training set in the sample data set;
s1.2.3 specifically comprises the steps of selecting a training platform and setting training iteration times, batch processing size, learning rate and attenuation rate; initializing the weight and the deviation parameter by a random function, and optimizing after each iteration;
and in the first full-connection layer, dropout is used, the performance of the GPU is improved by utilizing a cuDNN acceleration library, a ReLU function is adopted for nonlinear activation, and an Adam optimizer is used for cross entropy loss optimization.
Converting the one-hot coded labels into maximum normalized probability distribution so that the target level has the highest probability value; given a set of label ranks R = { R = { [ R ] QF-BKX ,r QF-KX ,r QF-YX ；r YQF-BKX ,r YQF-KX ,r YQF-YX ；r ZQF-BKX ,r ZQF-KX ,r ZQF-YX ；r YF-BKX ,r YF-KX ,r YF-YX ；r ZF-BKX ,r ZF-KX ,r ZF-YX R according to the target grade r t Prediction level r i Generating a ground truth value label y i Optimizing the training direction of the scene classification model: as shown in formula (3):
wherein r is i For the ith prediction level, r k For the kth prediction level, y i For the ith ground truth value tag,
is a metric function which penalizes the deviation of the predicted grade from the target grade, as shown in the formula (4)
When r i -r t |>N, y is reduced to one-hot encoded vector; when r i -r t |<When epsilon, y tends to be a uniform probability distribution;
given a pixel location p = [ px, py ] in a ground truth mask] T Calculating a weight map w (p), the loss applied to each pixel by element-level multiplication, the weight of a pixel depending on its euclidean distance d (p) to the nearest segmentation boundary and its vertical position h (p) in the image, as shown in equation (5):
wherein, β is a constant adjustable according to practical conditions, h (p) is used for scaling the rate of pixel weight increase when the pixel is far away from the boundary, and is used as a multiplication factor of pixel level to enable the pixel point of low pixel to obtain high weight.
The S2 comprises the following steps:
s2.1: vehicle motion data of a vehicle-mounted inertial sensor at the current moment and vehicle position and speed measurement data of a satellite are collected,
s2.2: establishing a system state space model, wherein the system state space model comprises a state equation model and a measurement equation model:
the equation of state model is as shown in equation (6):
X＝[ψ,δV,δ,γ,ζ] T (6)
wherein Ψ, V, δ, γ and ζ are represented by formula (7):
where ψ represents a vehicle attitude error variable, ψ E ,ψ N ,ψ U Respectively representing the attitude errors of the east, north and sky vehicles, delta V representing the speed error variable of the vehicle, delta V E ,δV N ,δV U Respectively representing east, north and sky vehicle speed errors, delta representing vehicle position error, delta L, delta lambdaAnd δ h respectively represent the dimension, precision and elevation error of the position of the vehicle, γ represents the offset of the accelerometer in the coordinate system of the body, γ bx ,、γ by 、γ bz The bias of the accelerometer in the X direction, the Y direction and the Z direction under the body coordinate system respectively, zeta represents the offset of the gyroscope under the body coordinate system, zeta bx ,、ζ by ,、ζ bz Respectively drift of the gyroscope in X, Y and Z directions under a body coordinate system;
the measurement equation model is represented by formula (8):
wherein, P state 、P gps Position data, v, representing vehicle-mounted inertial sensors and satellite measurements, respectively state 、v gps Respectively representing the speed data measured by the vehicle-mounted inertial sensor and the satellite, wherein the measurement variable Z is a matrix vector formed by the difference values of the position and the speed measured by the vehicle-mounted inertial sensor and the satellite;
s2.3: reading in the measurement information of the vehicle-mounted inertial sensor, and performing the prediction step of the extended Kalman filtering by using the constructed state equation model so as to obtain the prior estimation of the system state variable at the current moment
S2.4: prior state covariance matrix P k|k-1 And (3) self-adaptive estimation:
calculating filtering innovation v k ：
Wherein z is k Measurement data for a satellite;
calculating an adaptive parameter lambda k ：
Wherein R is k For observing the noise covariance matrix, Q k Is a state noise covariance matrix, P k-1 Is a covariance matrix of posterior states at time k-1, H k In order to observe the matrix, the system is,
is H k Transpose of matrix, parameter C 0 Is determined by the formula (11)
Wherein, the first and the second end of the pipe are connected with each other,
is v 0 Transpose of (v) 0 The filtering information matrix at the initial moment is given by experience according to the actual situation;
calculating a prior state covariance matrix at the current moment:
wherein, F k|k-1 Representing a state transition matrix at the k moment;
is F k|k-1 Transposing;
s2.5: reading in measurement information z of the current satellite k And performing an updating step of the extended Kalman filtering by using the constructed measurement equation model so as to correct the state variable of the system estimated a priori to obtain a posterior estimation result of the state variable of the system
The S3 comprises the following steps:
s3.1: according to the scene classification result, calculating control selection parameters:
the scene classification result is expressed as shown in equation (13):
wherein, the function theta is used for a convolutional neural network model of scene perception and outputs P QF-YX 、P QF-KX 、P QF-BKX 、P YQF-YX 、P YQF-KX 、P YQF-BKX 、P ZQF-YX 、P ZQF-KX、 P ZQF-BKX 、P YF-YX 、P YF-KX 、P YF-BKX 、P ZF-YX 、P ZF-KX And P ZF-BKX Fifteen kinds of probabilities are provided, wherein QF, YQF, ZQF, YF and ZF respectively represent right front, left front, right side and left side, and YX, KX and BKX respectively represent prior passing, feasible and infeasible; a is a model parameter, o t Is the currently observed image, d i Is the KL distance used to select models of different complexity;
defining a control selection parameter P QF 、P ZF 、P YF As shown in formulas (14), (15), and (16):
P QF ＝P QF-YX +P QF-KX +P QF-BKX (14)
P ZF ＝P ZF-YX +P ZF-KX +P ZF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (15)
P YF ＝P YF-YX +P YF-KX +P YF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (16)
wherein a is an adjustable parameter in the range of 0.5-0.8;
s3.2: selecting a control mode according to the control selection parameters, and controlling the vehicle to move:
wherein the functionπ AWC And pi RC Are two control modes, AWC for weighted control mode, RC for regular control mode, B for manually set baseline speed, [ A, B]The range of (c) is given empirically; velocity commands u = (v, w) include linear and angular velocities;
for the normal control mode, the linear velocity and the angular velocity at which the vehicle is controlled to travel are represented by equations (18), (19);
v＝V l P QF (18)
w＝Ω a (P ZF -P YF ) (19)
wherein the linear velocity v is P QF And a base line velocity V l The product of (a); angular velocity w of P ZF And P YF Difference between the base angular velocity omega a The product of (a);
for the weighted control mode, the forward speed is reduced and the turn speed is increased using adjustable empirical parameters.
The S4 comprises the following steps:
s4.1: acquiring current scene information by using a vehicle-mounted camera, and positioning a vehicle at the current moment by using the vehicle positioning at the current moment;
s4.2: according to a Kullback-Leibler (KL) distance method, selecting a scene classification model with high, medium and low complexity:
wherein p represents the posterior probability of the current driving environment category,
and
representing the posterior probability density of low, medium and high complexity, d i Is KL distance, E p Is the expectation function, i is the scene complexity level; the current driving scene complexity level i is according to d in formula (19) i Determining the minimum value, and selecting a model corresponding to the scene level complexity;
s4.3: classifying the current scene by using a scene classification model with corresponding complexity, and outputting fifteen scene probability value results by using the scene classification model;
s4.4: judging whether the driving decision is feasible or not according to the navigation task, the surrounding environment and the traffic regulations; if the driving decision is feasible, selecting a vehicle control mode by using a scene classification model output result, and generating an angular speed and a driving speed instruction for controlling steering;
if the driving decision is not feasible, generating prompt information and sending the prompt information to a user, wherein the prompt information comprises the reason why the driving decision is not feasible and the driving decision for recommending replacement;
s4.5: judging whether the vehicle reaches the key point or not according to the self-adaptive positioning result, and if the vehicle reaches the terminal point, obtaining a navigation result; if the terminal is not reached, returning to S3.1 to continue navigation.
Advantageous technical effects
1. The invention provides a low-cost navigation method based on an artificial intelligence technology, which aims to solve the problem of autonomous navigation of the existing automatic driving vehicle. The method utilizes a scene classification model based on a convolutional neural network field to understand the surrounding environment of the vehicle, and forms an end-to-end automatic driving vehicle navigation method through direct mapping of a scene classification result and vehicle control, does not need a complex intermediate module, avoids the difficulties of perception algorithm fusion, map construction and the like, greatly reduces the complexity of the system, and has the advantages of small development difficulty and low hardware requirement;
2. the invention provides various schemes to enhance the precision and robustness of the navigation method: an adaptive positioning module is provided to assist in judging the navigation process; different vehicle control modes are introduced; providing scene classification models suitable for different scene complexity, and providing a corresponding model selection method; a two-stage scene picture processing method is provided; different model training enhancement methods are introduced.
Drawings
FIG. 1 is a flowchart of a method for navigating an autonomous vehicle based on scene classification according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a scene classification model according to an embodiment of the present invention;
FIG. 3 is a schematic view of a vehicle driving control diagram provided by an embodiment of the invention;
FIG. 4 is a diagram of an autonomous vehicle navigation framework according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions and advantages thereof more clear, the technical solutions in the present application will be described below with reference to the accompanying drawings.
As shown in fig. 1, the method for navigating an automatic driving vehicle based on scene classification according to the present embodiment includes the following steps:
s1: scene classification, namely acquiring and classifying images of the surrounding environment of the automatic driving vehicle by using a vehicle-mounted camera; establishing a scene classification model based on a convolutional neural network technology, and training the scene classification model; the method specifically comprises the following steps:
s1.1: acquiring a sample data set, shooting scene pictures of different environments by using a vehicle-mounted camera, and classifying the scene pictures;
s1.2: establishing a scene classification model based on a convolutional neural network architecture, carrying out normalization processing on a data set, and training the established scene classification model by using a training set;
s1.3: adjusting the hyper-parameters of the scene classification model according to different scene complexity, wherein the hyper-parameters comprise learning rate, attenuation rate, iteration times, layer number, the number of neurons in each layer, batch size and weight of each part in a loss function; repeating S1.2, and training scene classification models suitable for different complexities, including high, medium and low complexity scene classification models;
the difference of the scene complexity represents the difference of the number of things contained in the scene picture in the sample data set; the complexity is low when the number of the object types is less than or equal to three, the complexity is medium when the number of the object types is more than three and less than seven, and the complexity is high when the number of the object types is more than seven;
the S1.1 specifically comprises the following steps:
s1.1.1: manually controlling the automatic driving vehicle to run in an outdoor non-structural environment, acquiring an environment image through a twistable camera arranged at the front end of the vehicle, generating a sample data set, and recording the shooting direction of each sample data; the twistable camera represents that the camera is provided with a fixed base, and the camera can twist plus and minus eighty degrees around the fixed base; the sample data shooting direction is divided into five directions, namely a front direction, a left front direction, a right front direction, a left direction and a right direction; the right front represents that the torsion angle of the camera is plus or minus ten degrees, the left front is more than plus ten degrees and less than plus sixty degrees, the right front is more than minus ten degrees and less than minus sixty degrees, the left is more than plus sixty degrees and less than plus eighty degrees, and the right is more than minus sixty degrees and less than minus eighty degrees;
s1.1.2: performing semantic classification on each sample scene picture in the sample data set, and selecting a scene classification label for each sample scene picture according to an actual scene;
the semantic classification represents classifying things in the sample scene picture according to categories, including ten categories of highways, non-motor vehicle roads, pedestrians, vehicles, buildings, street lamps, trees, grasslands, sand lands and sky; the scene classification labels comprise fifteen label categories, namely three conditions of priority, feasibility and infeasibility respectively corresponding to the right front, the left front, the right front, the left side and the right side; the priority indicates a place where the vehicle is expected to travel, i.e., an open road without obstacles; the feasibility represents a place where the vehicle may travel but is generally not desired to travel, such as non-motorized roadways, grass, and sand; the impracticable representation of places where vehicles cannot travel and things that should be avoided, such as buildings, street lights, trees, the sky, and roads with pedestrians and other obstacles;
s1.1.3: and (3) the sample data set after scene classification processing is proportioned to 8:2, dividing the training set into a training set and a testing set;
fifteen labels are shown in table 1:
TABLE 1
The S1.2 specifically comprises the following steps:
s1.2.1: establishing a scene classification model based on a shallow convolutional neural network architecture by using the existing Tensorflow open source framework as shown in FIG. 2; the model consists of three convolution layers, three pooling layers and two full-connection layers; outputting 32 feature graphs by each convolution and pooling layer, performing feature fusion through a full connection layer, and dividing a scene into fifteen probability values by utilizing a Softmax function to output; as shown in formula (1):
wherein z is i Outputting a value for the ith node of the full connection layer; z is a radical of formula c Outputting a value for the c-th node of the full connection layer;
s1.2.2: adjusting the size of a training set sample subjected to scene classification processing to 64 × 64 as the input of a scene classification model, and performing unified normalization processing on the training set and the test set:
x norm ＝(x RGB -min)/(max-min)-0.5 (2)
wherein x is RGB Color values for three channels; max and min are the extreme values of the current image, set to 255 and 0, respectively; the pixel intensity of each image is adjusted to [0.5,0.5]；
S1.2.3: training a scene classification model by using a training set in the sample data set:
using a NvidiaGTX1060GPU as a platform to train the model;
performing 1000 iterations to train the model, wherein the batch processing size is 1024 frames, the initial learning rate is set to be 0.001, and the attenuation is 0.05 every 10 steps; initializing the weight and the deviation parameter by a random function, and optimizing after each iteration;
in order to avoid overfitting as much as possible, dropout is used in the first full-connection layer, and a cuDNN acceleration library is used for improving GPU performance; in addition, a ReLU function is adopted for nonlinear activation, the problem of gradient disappearance is solved, and the training efficiency is improved; performing cross entropy loss optimization by using an Adam optimizer to relieve the problems of gradient sparsity and large noise;
converting the one-hot coded labels into maximum normalized probability distribution according to the sequencing definition so as to enable the target level to have the highest probability value; given a set of label ranks R = { R = { [ R ] QF-BKX ,r QF-KX ,r QF-YX ；r YQF-BKX ,r YQF-KX ,r YQF-YX ；r ZQF-BKX ,r ZQF-KX ,r ZQF-YX ；r YF-BKX ,r YF-KX ,r YF-YX ；r ZF-BKX ,r ZF-KX ,r ZF-YX R according to the target grade r t Generating a ground true value label y, and optimizing the training direction of the model: as shown in formula (3):
wherein r is i For the ith prediction level, r k For the k-th prediction level, y i For the ith ground truth value tag,
is a metric function which penalizes the deviation of the predicted grade from the target grade, as shown in the formula (4)
When the inter-level distance approaches infinity, y is reduced to one-hot encoded vector; when the distance tends to 0, y tends to be a uniform probability distribution;
the concept of image depth is introduced to distinguish near and distant elements; one pixel location p = [ px, py ] in a given ground truth mask] T Calculating a weight map w (p) which applies to the penalty of each pixel by element-level multiplication; the weight of a pixel depends on its euclidean distance d (p) to the nearest segmentation boundary and its vertical position h (p) in the image;
as shown in formula (5):
wherein β is a constant that is adjustable according to the actual situation; the height map h (p) is used to scale the rate at which the pixel weights increase as one moves away from the boundary and to assign higher weights to lower pixels as a multiplicative factor at the pixel level; it is used as the original placeholder for depth data, and the working principle is that the lower the pixel in the image is, the closer it is to the camera;
s2: based on an extended Kalman filtering framework, establishing a system state model, predicting prior estimation of a stepping state variable by using extended Kalman filtering, updating a covariance matrix of the prior state in real time by using a filtering innovation-based adaptive algorithm, and updating posterior estimation of the stepping state variable by using the extended Kalman filtering to realize vehicle adaptive positioning; the method specifically comprises the following steps:
s2.1: vehicle motion data of a vehicle-mounted inertial sensor at the current moment and vehicle position and speed measurement data of a satellite are collected,
s2.2: establishing a system state space model, wherein the system state space model comprises a state equation model and a measurement equation model:
the state equation model is represented by equation (6):
X＝[ψ,δV,δ,γ,ζ] T (6)
wherein Ψ, V, δ, γ and ζ are represented by formula (7):
wherein ψ represents a vehicle attitude error variable, ψ E ,ψ N ,ψ U Respectively representing the attitude errors of the east, north and sky vehicles, delta V representing the speed error variable of the vehicle, delta V E ,δV N ,δV U Respectively representing east, north and sky vehicle speed errors, delta representing vehicle position error, delta L, delta lambda and delta h respectivelyError of dimension, precision and elevation of the position of the vehicle is represented, gamma represents offset of the accelerometer under a body coordinate system, gamma represents offset of the accelerometer under the body coordinate system bx ,、γ by 、γ bz Bias of the accelerometer in X, Y and Z directions in a body coordinate system, zeta represents offset of the gyroscope in the body coordinate system, and zeta represents offset of the gyroscope in the body coordinate system bx ,、ζ by ,、ζ bz Respectively drift of the gyroscope in X, Y and Z directions under a body coordinate system;
the measurement equation model is shown in formula (8):
wherein, P state 、P gps Position data, v, representing vehicle-mounted inertial sensors and satellite measurements, respectively state 、v gps Respectively representing the speed data measured by the vehicle-mounted inertial sensor and the satellite; the measurement variable Z is a matrix vector formed by the difference values of the position and the speed measured by the vehicle-mounted inertial sensor and the satellite;
s2.3: reading in measurement information of the vehicle-mounted inertial sensor, and performing prediction step of extended Kalman filtering by using the constructed state equation model so as to obtain prior estimation of the state variable of the system at the current moment
S2.4: prior state covariance matrix P k|k-1 Self-adaptive estimation:
calculating a filtering innovation v k
Wherein z is k Measurement data for a satellite;
calculating an adaptive parameter lambda k
Wherein R is k For observing the noise covariance matrix, Q k Is a state noise covariance matrix, P k-1 Is a covariance matrix of posterior states at time k-1, H k For the observation matrix, parameter C 0 Is determined by the formula (11)
Wherein the content of the first and second substances,
is v is 0 Transpose of v 0 The filtering information matrix at the initial moment is given by experience according to the actual situation;
calculating prior state covariance matrix at current moment
Wherein, F k|k-1 Representing a state transition matrix at the k moment;
s2.5: reading in measurement information z of the current satellite k And performing the updating step of the extended Kalman filtering by using the constructed measurement equation model so as to correct the state variable of the system estimated a priori to obtain the posterior estimation result of the state variable of the system
S3: as shown in fig. 3, the vehicle travel control is performed by selecting a vehicle control mode according to the classification result of the scene classification model; acquiring surrounding environment information through a vehicle, and selecting a corresponding scene classification model to perform multi-modal understanding on the surrounding environment information of the vehicle according to the number of the types of objects in the scene, so as to determine a driving decision according to a scene classification result; then according to the requirement of the navigation task, generating a steering and speed instruction for the vehicle, so that the automatic driving vehicle can automatically identify the emergency situation according to the understanding of the scene when executing the navigation task, and accurately give a control instruction for the vehicle; the method specifically comprises the following steps:
s3.1: according to the scene classification result, calculating a control selection parameter:
the scene classification result is expressed as shown in equation (13):
the function theta is used for a convolutional neural network model for scene perception, fifteen probabilities are output, a is a model parameter, and o is t Is the image that is currently being observed and,
defining a control selection parameter P QF 、P ZF 、P YF As shown in formulas (14), (15), and (16):
P QF ＝P QF-YX +P QF-KX +P QF-BKX (14)
P ZF ＝P ZF-YX +P ZF-KX +P ZF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (15)
P YF ＝P YF-YX +P YF-KX +P YF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (16)
wherein a is an adjustable parameter in the range of 0.5-0.8;
s3.2: selecting a control mode according to the control selection parameters, and controlling the vehicle to move:
because in some special cases, the vehicle can sense the obstacles around, but the fifteen probability values are all small and are not enough to drive the vehicle away from nearby objects; aiming at the problem, the method is used in a navigation system by combining an Adaptive Weighted Control (AWC) algorithm and a conventional control (RC) algorithm; the AWC algorithm adds a self-adaptive adjustment factor according to an environment understanding result, and drives the vehicle to run in a corresponding direction and at a corresponding speed; the specific control mode is selected as the formula (17):
wherein the function pi AWC And pi RC Are two control modes, AWC for weighted control mode, RC for regular control mode, B for manually set baseline speed, [ A, B]The range of (a) is given empirically; velocity command u = (v, w) includes linear velocity and angular velocity;
for the normal control mode, the linear velocity and the angular velocity at which the vehicle is controlled to travel are represented by equations (18), (19);
v＝V l P QF (18)
w＝Ω a (P ZF -P YF ) (19)
wherein the linear velocity v is P QF And a base line velocity V l The product of (a); angular velocity w of P ZF And P YF Difference between the base angular velocity omega a The product of (a);
wherein the linear velocity is P QF Product of the baseline velocity; angular velocity P ZF And P YF The product of the difference and the base angular velocity;
for the weighted control mode, the advancing speed is reduced and the turning speed is increased by using the adjustable empirical parameters;
s4: and establishing an automatic driving automobile navigation frame according to the S1, the S2 and the S3, wherein as shown in the figure 4, the navigation process of the navigation frame comprises the following steps: collecting surrounding environment images of the automatic driving vehicle at the current moment as input of a scene classification model, carrying out driving control on the vehicle according to the output of the scene classification model, carrying out position judgment on the current vehicle according to a self-adaptive positioning result, and iterating until the automatic driving vehicle reaches a target position;
in this embodiment, the automatic driving car navigation model is established: the navigation model consists of a sensor, a control mode decision, a controller and self-adaptive positioning; the input of the navigation model is a processed current scene picture of the automatic driving vehicle; the sensor is responsible for image classification and generates P respectively QF-YX 、P QF-KX 、P QF-BKX 、P YQF-YX 、P YQF-KX 、P YQF-BKX 、P ZQF-YX 、P ZQF-KX 、P ZQF-BKX 、P YF-YX 、P YF-KX 、P YF-BKX 、P ZF-YX 、P ZF-KX And P ZF-BKX Fifteen probabilities are used as input of control mode decision; the probability value corresponds to the feasible direction of the vehicle motion, and indirectly determines the control mode and the speed instruction of the vehicle; the self-adaptive positioning is used for determining the position of the vehicle and judging the navigation progress; the method specifically comprises the following steps:
s4.1: activating a camera, acquiring the current environment information around the vehicle by using the vehicle-mounted camera, and positioning the vehicle at the current moment by using the self-adaptive positioning scheme S2: the method comprises the steps that a camera is activated, namely a shooting activation signal is sent to the camera to activate the camera to shoot environmental information around a vehicle; acquiring environmental information around the vehicle includes: acquiring environmental information around the vehicle, which is shot by a shooting device according to a shooting activation signal, wherein the environmental information shot by a camera is image information; acquiring image information of an environment around a vehicle includes: acquiring environmental information around a vehicle, which is periodically photographed by a camera;
s4.2: according to a Kullback-Leibler (KL) distance method, selecting a scene classification model with high, medium and low complexity:
where p represents the posterior probability of the current driving environment category, i represents a different category of complexity,
and
representing the posterior probability density of low, medium and high complexity, d i Is a KL distance, E p Is the expectation function, i is the scene complexity level; the current driving scene complexity level i can be determined according to d in formula (19) i Minimum value, selecting the scene with complex corresponding scene levelA model;
wherein p represents the posterior probability of the current driving environment class, p1class, p2class and p3class represent the posterior probability density of low, medium and high complexity respectively, and d i Is a KL distance, E p Is the expectation function, i is the scene complexity level; the current driving scene complexity level i can be determined according to d in formula (19) i Determining the minimum value, and selecting a model with complex corresponding scene grade;
s4.3: classifying the current scene by using a scene classification model with corresponding complexity, and outputting fifteen scene probability value results by using the scene classification model;
s4.4: judging whether the driving decision is feasible or not according to the navigation task, the surrounding environment and the traffic regulations; if the driving decision is feasible, selecting a vehicle control mode by using a scene classification model output result, and generating an angular speed and a driving speed instruction for controlling steering;
if the driving decision is not feasible, generating prompt information and sending the prompt information to a user, wherein the prompt information comprises the reason why the driving decision is not feasible and the driving decision for recommending replacement;
s4.5: judging whether the vehicle reaches the key point or not according to the self-adaptive positioning result, and if the vehicle reaches the terminal point, obtaining a navigation result; if the terminal is not reached, returning to S3.1 to continue navigation.
Claims (10)
1. An automatic driving vehicle navigation method based on scene classification is characterized in that: the method comprises the following steps:
s1: acquiring and classifying images of the surrounding environment of the automatic driving vehicle by using a vehicle-mounted camera; establishing a scene classification model based on a convolutional neural network technology, and training the scene classification model;
s2: based on an extended Kalman filtering framework, a system state model is established, the prior estimation of the stepping state variable is predicted by using extended Kalman filtering, a covariance matrix of the prior state is updated in real time by using an adaptive algorithm based on filtering information, the posterior estimation of the stepping state variable is updated by using extended Kalman filtering, the adaptive positioning of the vehicle is realized,
s3: selecting a vehicle control mode according to the classification result of the scene classification model to control the running of the vehicle;
s4: establishing an automatic driving automobile navigation frame; the navigation process of the navigation framework comprises the following steps: and collecting the surrounding environment image of the automatic driving vehicle at the current moment as the input of a scene classification model, carrying out the driving control of the vehicle according to the output of the scene classification model, judging the current vehicle position according to the self-adaptive positioning result, and repeating the steps until the automatic driving vehicle reaches the target position.
2. The method of scene classification based autonomous vehicle navigation according to claim 1, characterized in that:
the S1 specifically comprises the following steps:
s1.1: acquiring a sample data set: shooting scene pictures of different environments by using a vehicle-mounted camera, and classifying the scene pictures;
s1.2: establishing a scene classification model based on a convolutional neural network architecture, carrying out normalization processing on a data set, and training the established scene classification model by using a training set;
s1.3: adjusting the hyper-parameters of the scene classification model according to the number of the object types in the scene, wherein the hyper-parameters comprise a learning rate, an attenuation rate, iteration times, the number of layers, the number of neurons in each layer, batch size and the weight of each part in a loss function; and repeating the S1.2, and training the scene classification models suitable for different complexities, including scene classification models with high, medium and low complexities.
3. The method of scene classification-based autonomous vehicle navigation according to claim 2, characterized in that:
the difference of the scene complexity represents the difference of the number of the object types contained in the scene picture in the sample data set; the complexity is low when the number of the object types is less than or equal to three, medium complexity when the number of the object types is more than three and less than seven, and high complexity when the number of the object types is more than seven.
4. The method of claim 1, wherein the method comprises:
the S1.1 specifically comprises the following steps:
s1.1.1: manually controlling the vehicle to run, acquiring an environment image through a camera arranged at a preset position of the vehicle, generating a sample data set, and recording the shooting direction of each sample data;
the camera can be twisted by plus or minus eighty degrees; the sample data shooting direction is divided into five directions, namely a front direction, a left front direction, a right front direction, a left direction and a right direction;
s1.1.2: performing semantic classification on each sample scene picture in the sample data set, and selecting a scene classification label for each sample scene picture according to an actual scene;
the semantic classification represents classifying things in the sample scene picture according to categories, including ten categories of highways, non-motor vehicle roads, pedestrians, vehicles, buildings, street lamps, trees, grasslands, sand lands and sky; the scene classification labels comprise fifteen label categories, namely, three conditions of priority, feasibility and infeasibility respectively correspond to the right front, the left front, the right front, the left side and the right side;
s1.1.3: and dividing the sample data set subjected to scene classification into a training set and a test set according to a proportion.
5. The method of scene classification based autonomous vehicle navigation according to claim 1, characterized in that:
the S1.2 specifically comprises the following steps:
s1.2.1: establishing a scene classification model based on a shallow convolutional neural network architecture by using the existing Tensorflow open source framework; the model comprises a convolution layer, a pooling layer and a full-connection layer; outputting x characteristic graphs by each convolution and pooling layer, performing characteristic fusion by a full connection layer, and dividing a scene into fifteen probability values by utilizing a Softmax function to output; as shown in formula (1):
wherein z is i Outputting a value for the ith node of the full connection layer; z is a radical of formula c Outputting a value for the c node of the full connection layer;
s1.2.2: adjusting the training set sample subjected to scene classification processing into y x y size as the input of a scene classification model, and performing unified normalization processing on the training set and the test set:
x norm ＝(x RGB -min)/(max-min)-b (2)
wherein x is RGB Color values for three channels; max and min are the extreme values of the current image, set to 255 and 0, respectively; adjust the pixel intensity of each image to [ b, b]；
S1.2.3: and training the scene classification model by using a training set in the sample data set.
6. The method of scene classification-based autonomous vehicle navigation of claim 5, characterized in that:
s1.2.3 specifically comprises the steps of selecting a training platform and setting training iteration times, batch processing size, learning rate and attenuation rate; initializing the weight and the deviation parameter by a random function, and optimizing after each iteration;
using dropout in a first full-connection layer, improving the performance of a GPU by using a cuDNN acceleration library, carrying out nonlinear activation by using a ReLU function, and carrying out cross entropy loss optimization by using an Adam optimizer;
converting the one-hot coded labels into maximum normalized probability distribution so that the target level has the highest probability value; given a set of label ranks R = { R = { [ R ] QF-BKX ,r QF-KX ,r QF-YX ；r YQF-BKX ,r YQF-KX ,r YQF-YX ；r ZQF-BKX ,r ZQF-KX ,r ZQF-YX ；r YF-BKX ,r YF-KX ,r YF-YX ；r ZF-BKX ,r ZF-KX ,r ZF-YX R according to the target grade r t Prediction level r i Generating a ground truth value label y i And optimizing the model training direction: as shown in formula (3):
wherein r is i For the ith prediction level, r k For the k-th prediction level, y i For the ith ground truth value tag,
is a metric function for punishing the deviation of the prediction grade from the target grade, as shown in the formula (4):
when r i -r t |>N, y is reduced to one-hot encoded vector; when r i -r t |<When epsilon, y tends to be a uniform probability distribution;
given a pixel location p = [ px, py ] in a ground truth mask] T Calculating a weight map w (p), the loss applied to each pixel by element-level multiplication, the weight of a pixel depending on its euclidean distance d (p) to the nearest segmentation boundary and its vertical position h (p) in the image, as shown in equation (5):
wherein, β is a constant adjustable according to practical conditions, h (p) is used for scaling the rate of pixel weight increase when the boundary is far away, and is used as a multiplication factor of pixel level to enable the pixel point of low pixel to obtain high weight.
7. The method of scene classification based autonomous vehicle navigation according to claim 1, characterized in that:
the S2 comprises the following steps:
s2.1: collecting vehicle motion data of a vehicle-mounted inertial sensor at the current moment and vehicle position and speed measurement data of a satellite,
s2.2: establishing a system state space model, wherein the system state space model comprises a state equation model and a measurement equation model;
s2.3: reading in measurement information of the vehicle-mounted inertial sensor, and performing prediction step of extended Kalman filtering by using the constructed state equation model so as to obtain prior estimation of the state variable of the system at the current moment
S2.4: prior state covariance matrix P k|k-1 And (3) self-adaptive estimation:
calculating filtering innovation v k
Wherein z is k Measurement data for a satellite;
calculating an adaptive parameter lambda k
Wherein R is k To observe the noise covariance matrix, Q k Is a state noise covariance matrix, P k-1 Is a covariance matrix of posterior states at time k-1, H k In order to observe the matrix, the system,
is H k Transpose of matrix, parameter C 0 Is determined by the formula (11)
Wherein the content of the first and second substances,
is v is 0 Transpose of v 0 The filtering information matrix at the initial moment is given by experience according to the actual situation;
calculating prior state covariance matrix at current moment
Wherein, F k|k-1 Representing a state transition matrix at the k moment;
is F k|k-1 Transposing;
s2.5: reading in measurement information z of the current satellite k And performing an updating step of the extended Kalman filtering by using the constructed measurement equation model so as to correct the state variable of the system estimated a priori to obtain a posterior estimation result of the state variable of the system
8. The method of scene classification-based autonomous vehicle navigation of claim 7, characterized in that:
the state equation model is represented by equation (6):
X＝[ψ,δV,δ,γ,ζ] T (6)
wherein Ψ, V, δ, γ and ζ are represented by formula (7):
where ψ represents a vehicle attitude error variable, ψ E ,ψ N ,ψ U Respectively representing the attitude errors of the east, north and sky vehicles, delta V representing the speed error variable of the vehicle, delta V E ,δV N ,δV U Respectively representing east, north and sky vehicle speed errors, delta representing a vehicle position error, delta L, delta lambda and delta h representing dimensions, precision and elevation errors of the position of the vehicle, gamma representing the offset of the accelerometer in a body coordinate system, gamma bx ,、γ by 、γ bz The bias of the accelerometer in the X direction, the Y direction and the Z direction under the body coordinate system respectively, zeta represents the offset of the gyroscope under the body coordinate system, zeta bx ,、ζ by ,、ζ bz Respectively drift of the gyroscope in X, Y and Z directions under a body coordinate system;
the measurement equation model is shown in formula (8):
wherein, P state 、P gps Position data, v, representing vehicle-mounted inertial sensors and satellite measurements, respectively state 、v gps The speed data measured by the vehicle-mounted inertial sensor and the satellite are respectively represented, and the measurement variable Z is a matrix vector formed by the difference values of the position and the speed measured by the vehicle-mounted inertial sensor and the satellite.
9. The method of scene classification based autonomous vehicle navigation according to claim 1, characterized in that:
the S3 comprises the following steps:
s3.1: according to the scene classification result, calculating control selection parameters:
the scene classification result is expressed as shown in equation (13):
wherein, the function theta is used for a convolutional neural network model of scene perception and outputs P QF-YX 、P QF-KX 、P QF-BKX 、P YQF-YX 、P YQF-KX 、P YQF-BKX 、P ZQF-YX 、P ZQF-KX 、P ZQF-BKX 、P YF-YX 、P YF-KX 、P YF-BKX 、P ZF-YX 、P ZF-KX And P ZF-BKX Fifteen kinds of probabilities are provided, wherein QF, YQF, ZQF, YF and ZF respectively represent right front, left front, right side and left side, and YX, KX and BKX respectively represent prior passing, feasible and infeasible; a is a model parameter, o t Is the currently observed image, d i Is the KL distance used to select models of different complexity;
defining a control selection parameter P QF 、P ZF 、P YF As shown in formulas (14), (15), and (16):
P QF ＝P QF-YX +P QF-KX +P QF-BKX (14)
P ZF ＝P ZF-YX +P ZF-KX +P ZF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (15)
P YF ＝P YF-YX +P YF-KX +P YF-BKX +a(P ZQF-YX +P ZQF-KX +P ZQF-BKX ) (16)
wherein a is an adjustable parameter in the range of 0.5-0.8;
s3.2: selecting a control mode according to the control selection parameters, and controlling the vehicle to move:
wherein the function pi AWC And pi RC Are two control modes, AWC for weighted control mode, RC for regular control mode, B for manually set baseline speed, [ A, B]The range of (a) is given empirically; velocity command u = (v, w) includes linear velocity and angular velocity;
for the normal control mode, the linear velocity and the angular velocity at which the vehicle is controlled to travel are represented by equations (18), (19);
v＝V l P QF (18)
w＝Ω a (P ZF -P YF ) (19)
wherein the linear velocity v is P QF And a base line velocity V l The product of (a); angular velocity w of P ZF And P YF Difference between the base angular velocity omega a The product of (a);
for the weighted control mode, the forward speed is reduced and the turn speed is increased using adjustable empirical parameters.
10. The method of claim 1, wherein the method comprises: the S4 comprises the following steps:
s4.1: acquiring current scene information by using a vehicle-mounted camera, and positioning a vehicle at the current moment by using the vehicle positioning at the current moment;
s4.2: according to a Kullback-Leibler (KL) distance method, selecting a scene classification model with high, medium and low complexity:
wherein p represents a posterior probability of the current driving environment category,
and
a posteriori probability densities of low, medium and high complexity, d i Is KL distance, E p Is the expectation function, i is the scene complexity level; the current driving scene complexity level i is according to (19) wherein d i Determining the minimum value, and selecting a model with complex corresponding scene grade;
s4.3: classifying the current scene by using a scene classification model with corresponding complexity, and outputting fifteen scene probability value results by using the scene classification model;
s4.4: judging whether the driving decision is feasible or not according to the navigation task, the surrounding environment and the traffic regulations; if the driving decision is feasible, selecting a vehicle control mode by using a scene classification model output result, and generating an angular speed and a driving speed instruction for controlling steering;
if the driving decision is not feasible, generating prompt information and sending the prompt information to a user, wherein the prompt information comprises the reason why the driving decision is not feasible and the driving decision for recommending replacement;
s4.5: judging whether the vehicle reaches the key point or not according to the self-adaptive positioning result, and if the vehicle reaches the terminal point, obtaining a navigation result; if the terminal is not reached, the navigation is continued by returning to S3.1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210925922.8A CN115285143A (en) | 2022-08-03 | 2022-08-03 | Automatic driving vehicle navigation method based on scene classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210925922.8A CN115285143A (en) | 2022-08-03 | 2022-08-03 | Automatic driving vehicle navigation method based on scene classification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115285143A true CN115285143A (en) | 2022-11-04 |
Family
ID=83825349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210925922.8A Pending CN115285143A (en) | 2022-08-03 | 2022-08-03 | Automatic driving vehicle navigation method based on scene classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115285143A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116013091A (en) * | 2023-03-24 | 2023-04-25 | 山东康威大数据科技有限公司 | Tunnel monitoring system and analysis method based on traffic flow big data |
| CN117574111A (en) * | 2024-01-15 | 2024-02-20 | 大秦数字能源技术股份有限公司 | BMS algorithm selection method, device, equipment and medium based on scene state |
| CN117975736A (en) * | 2024-03-29 | 2024-05-03 | 北京市计量检测科学研究院 | Unmanned vehicle road cooperative application scene test method and system |
| CN118097604A (en) * | 2024-04-24 | 2024-05-28 | 广汽埃安新能源汽车股份有限公司 | Intelligent vehicle environment sensing method and device |
| WO2024124446A1 (en) * | 2022-12-14 | 2024-06-20 | Siemens Aktiengesellschaft | Measurement method, apparatus, system and electronic device for a motion control system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154472A (en) * | 2017-11-30 | 2018-06-12 | 惠州市德赛西威汽车电子股份有限公司 | Merge the parking position visible detection method and system of navigation information |
| CN112232126A (en) * | 2020-09-14 | 2021-01-15 | 广东工业大学 | Dimension reduction expression method for improving variable scene positioning robustness |
| WO2021017212A1 (en) * | 2019-07-26 | 2021-02-04 | 魔门塔(苏州)科技有限公司 | Multi-scene high-precision vehicle positioning method and apparatus, and vehicle-mounted terminal |
| CN112868022A (en) * | 2018-10-16 | 2021-05-28 | 法弗人工智能有限公司 | Driving scenarios for autonomous vehicles |
| JP2022000376A (en) * | 2020-09-23 | 2022-01-04 | 阿波▲羅▼智▲聯▼（北京）科技有限公司 | Automatic drive function control method, device, electronic apparatus and storage medium |
| CN114415224A (en) * | 2022-01-25 | 2022-04-29 | 东北大学 | Vehicle fusion positioning system and method in complex limited environment |
-
2022
- 2022-08-03 CN CN202210925922.8A patent/CN115285143A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154472A (en) * | 2017-11-30 | 2018-06-12 | 惠州市德赛西威汽车电子股份有限公司 | Merge the parking position visible detection method and system of navigation information |
| CN112868022A (en) * | 2018-10-16 | 2021-05-28 | 法弗人工智能有限公司 | Driving scenarios for autonomous vehicles |
| WO2021017212A1 (en) * | 2019-07-26 | 2021-02-04 | 魔门塔(苏州)科技有限公司 | Multi-scene high-precision vehicle positioning method and apparatus, and vehicle-mounted terminal |
| CN112232126A (en) * | 2020-09-14 | 2021-01-15 | 广东工业大学 | Dimension reduction expression method for improving variable scene positioning robustness |
| JP2022000376A (en) * | 2020-09-23 | 2022-01-04 | 阿波▲羅▼智▲聯▼（北京）科技有限公司 | Automatic drive function control method, device, electronic apparatus and storage medium |
| CN114415224A (en) * | 2022-01-25 | 2022-04-29 | 东北大学 | Vehicle fusion positioning system and method in complex limited environment |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024124446A1 (en) * | 2022-12-14 | 2024-06-20 | Siemens Aktiengesellschaft | Measurement method, apparatus, system and electronic device for a motion control system |
| CN116013091A (en) * | 2023-03-24 | 2023-04-25 | 山东康威大数据科技有限公司 | Tunnel monitoring system and analysis method based on traffic flow big data |
| CN117574111A (en) * | 2024-01-15 | 2024-02-20 | 大秦数字能源技术股份有限公司 | BMS algorithm selection method, device, equipment and medium based on scene state |
| CN117574111B (en) * | 2024-01-15 | 2024-03-19 | 大秦数字能源技术股份有限公司 | BMS algorithm selection method, device, equipment and medium based on scene state |
| CN117975736A (en) * | 2024-03-29 | 2024-05-03 | 北京市计量检测科学研究院 | Unmanned vehicle road cooperative application scene test method and system |
| CN117975736B (en) * | 2024-03-29 | 2024-06-07 | 北京市计量检测科学研究院 | Unmanned vehicle road cooperative application scene test method and system |
| CN118097604A (en) * | 2024-04-24 | 2024-05-28 | 广汽埃安新能源汽车股份有限公司 | Intelligent vehicle environment sensing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115285143A (en) | Automatic driving vehicle navigation method based on scene classification | |
| KR102144623B1 (en) | Method for acquiring movement attributes of moving object and apparatus for performing the same | |
| CN110007675B (en) | Vehicle automatic driving decision-making system based on driving situation map and training set preparation method based on unmanned aerial vehicle | |
| Cai et al. | Probabilistic end-to-end vehicle navigation in complex dynamic environments with multimodal sensor fusion | |
| US11410315B2 (en) | High quality instance segmentation | |
| US20190145765A1 (en) | Three Dimensional Object Detection | |
| CN112212872B (en) | End-to-end automatic driving method and system based on laser radar and navigation map | |
| CN112734808B (en) | Trajectory prediction method for vulnerable road users in vehicle driving environment | |
| CN113044064B (en) | Vehicle self-adaptive automatic driving decision method and system based on meta reinforcement learning | |
| KR102296509B1 (en) | Method and device for segmenting image to be used for surveillance using weighted convolution filters for respective grid cells by converting modes according to classes of areas to satisfy level 4 of autonomous vehicle, and testing method and testing device using the same | |
| KR20200091318A (en) | Learning method and learning device for attention-driven image segmentation by using at least one adaptive loss weight map to be used for updating hd maps required to satisfy level 4 of autonomous vehicles and testing method and testing device using the same | |
| EP3686837B1 (en) | Learning method and learning device for reducing distortion occurred in warped image generated in process of stabilizing jittered image by using gan to enhance fault tolerance and fluctuation robustness in extreme situations | |
| CN113743469B (en) | Automatic driving decision method integrating multi-source data and comprehensive multi-dimensional indexes | |
| CN112651374B (en) | Future trajectory prediction method based on social information and automatic driving system | |
| CN113552883B (en) | Ground unmanned vehicle autonomous driving method and system based on deep reinforcement learning | |
| CN115049130B (en) | Automatic driving track prediction method based on space-time pyramid | |
| CN116182875A (en) | Temporary road path planning method and system based on graphic neural network | |
| CN115494849A (en) | Navigation control method and system for automatic driving vehicle | |
| CN114048536A (en) | Road structure prediction and target detection method based on multitask neural network | |
| WO2023155903A1 (en) | Systems and methods for generating road surface semantic segmentation map from sequence of point clouds | |
| CN114782915B (en) | Intelligent automobile end-to-end lane line detection system and equipment based on auxiliary supervision and knowledge distillation | |
| CN114895682B (en) | Unmanned mine car walking parameter correction method and system based on cloud data | |
| EP4124995A1 (en) | Training method for training an agent for controlling a controlled device, control method for controlling the controlled device, computer program(s), computer readable medium, training system and control system | |
| CN114779764A (en) | Vehicle reinforcement learning motion planning method based on driving risk analysis | |
| Zhang et al. | Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |