CN114926823A - WGCN-based vehicle driving behavior prediction method - Google Patents

Abstract

The invention relates to the technical field of vehicle driving behavior prediction, in particular to a WGCN-based vehicle driving behavior prediction method, which comprises the following steps: firstly, generating a characteristic matrix and a local map of each vehicle; secondly, constructing a map by the local map after weighting the feature matrix and encoding the local map by the CNN, and inputting the map into the GCN; thirdly, extracting the characteristics of input data through GCN, increasing the dimension of the edge characteristics by using an edge-enhanced attention mechanism of GCN, and improving the accuracy of weight coefficient distribution; the GCN characteristic transmission mechanism is utilized to enable the interactive characteristics of the vehicles to be transmitted in a graph form, and the change of the interactive relation between the vehicles is fully represented; inputting the interactive characteristics output by the GCN into a Transformer for training; and fifthly, obtaining a prediction result of the driving behavior of the vehicle through the full connection layer. The invention enables the vehicle driving behavior prediction to have higher accuracy.

Description

Prediction method of vehicle driving behavior based on WGCN

technical field

The invention relates to the technical field of vehicle driving behavior prediction, in particular to a WGCN-based vehicle driving behavior prediction method.

Background technique

With the development of vehicle networking technology and machine learning technology, in the field of vehicle driving, traditional assisted driving functions, such as adaptive cruise control (Adaptive Cruise Control, ACC) and lane keeping (Lane Keeping, LP), can no longer be To meet people's needs for intelligent vehicles, people urgently need a vehicle assisted driving system with richer functions and more intelligence. Therefore, autonomous driving technology has gradually entered people's field of vision. In order to realize the automatic driving of the vehicle, first of all, the vehicle needs to be equipped with many sensors such as millimeter wave radar, on-board image sensor, and global positioning system (Global Positioning System, GPS), in order to obtain real-time accurate own vehicle information and surrounding environment information. Secondly, the vehicle needs to be equipped with a powerful independent computing unit, such as Tesla's FSD (Full Self-Driving Computer, FSD), Huawei's MDC (Mobile Data Center, MDC), to quickly and accurately calculate the massive data. Finally, it needs to be combined with modern communication technology, so that the various information collected by the vehicle and the request of the vehicle can be quickly transmitted and responded. However, traditional autonomous vehicles are affected by factors such as the number of connected vehicles, road environment, traffic conditions, and computing power of computing units, making it difficult for vehicles to provide high quality of autonomous driving services. In addition, the communication resources of autonomous vehicles are limited, especially in complex traffic scenarios, which will cause a high delay in the reception and transmission of data by the vehicle, unable to process real-time data, and cause safety problems.

The emergence of mobile edge computing and deep learning can help solve the problem of insufficient computing and communication resources for autonomous driving, and can improve the intelligence of autonomous vehicles. Specifically, various tasks in automatic driving, such as vehicle recognition, vehicle behavior prediction, etc., can be implemented through deep learning, and can achieve high accuracy. A method based on Hidden Markov Chains is proposed in the literature. It can predict the lane-changing behavior within 0.5-0.7s; some literature and others obtain information such as accelerator, steering wheel, vehicle deflection angle through sensors, and achieve good results in driving behavior prediction on the ACT-R architecture; some literatures establish fuzzy The neural network combines the risk factor and the lane-changing feasibility factor to judge the driver's behavioral intention. The neural network model obtained through deep learning can be placed on the edge server, and the vehicle can obtain low-latency and high-precision autonomous driving services by requesting the edge server. Some literatures propose a real-time traffic prediction algorithm based on LSTM network. The algorithm extracts vehicle behavior characteristics by learning vehicle movement and interaction information, and obtains high prediction accuracy. Some literatures use multi-channel grid maps to represent traffic scenes, and use CNN to predict the driving trajectory of vehicles, and obtain good prediction results.

Although the current research on vehicle driving behavior prediction has made some progress, there are still two problems that need to be solved urgently. First, the existing research has not considered the interaction information between vehicles, and the key information of the interaction between vehicles is missing, which has a certain impact on the accuracy of prediction. Secondly, many current studies only consider the temporal and spatial characteristics of vehicles alone, and lack of research that combines temporal and spatial characteristics, making the characteristics of vehicle driving information insufficient.

SUMMARY OF THE INVENTION

The content of the present invention is to provide a WGCN-based vehicle driving behavior prediction method, which can overcome some or some defects of the prior art.

According to a WGCN-based vehicle driving behavior prediction method of the present invention, it comprises the following steps:

1. Generate the feature matrix and local map of each vehicle;

2. The weighted feature matrix and the local map encoded by CNN are constructed into a map and input into GCN;

3. Extract the features of the input data through GCN, and use the edge-enhanced attention mechanism of GCN to increase the dimension of edge features and improve the accuracy of weight coefficient allocation; use the feature transfer mechanism of GCN to make the interactive features of vehicles in the form of graphs. The form is transmitted, which fully expresses the change of the interaction relationship between the vehicles;

4. Input the interactive features output by GCN into Transformer for training;

5. Obtain the prediction result of vehicle driving behavior through the fully connected layer.

Preferably, in step 1, a feature matrix X is generated, X=[P,M], where P is a node feature matrix, including position (x, y), velocity (v _x , v _y ), and heading angle θ, M for the local map;

Preferably, in step 2, the edge-enhancement in the edge-enhancement attention mechanism is to increase the dimension of the edge feature, so that the edge feature can express more information; the attention mechanism is to assign different weight coefficients to different vertices. The weighted vertices have different priorities in processing. The higher the weight of the vertex, the richer the information of the vertex and the greater the influence.

As an option, in the edge-enhanced attention mechanism, vertex n and the edge feature vectors of surrounding vertices calculate the weights of surrounding vehicles, and the ultimate goal is to generate an adjacency matrix with weights to represent the influence between different vehicles. The adjacency matrix is represented as follows:

A'=softmax(A)

The above formula describes the process of the attention matrix A'. First, the edge feature matrix E needs to be normalized to obtain E', and then the edge feature matrix is multiplied by the trainable attention parameter matrix W _a to obtain the attention matrix A, Then perform softmax calculation on the matrix A to obtain A', so that the value of the elements in A' is in the range of 0 to 1, so as to express different weights; finally, the weighted adjacency matrix A _adj is obtained:

A _adj =E'A'.

Preferably, in the feature transfer mechanism, the adjacency relationship and interaction features between vehicles are used as the input of the neural network in the form of a graph, and the feature transfer method of the graph convolutional neural network is used to update the information, so that the network can fully extract the relationship between the vehicles. internal relationship;

The feature matrix X and the weighted adjacency matrix A _adj are used as update information to propagate in the constructed interaction model. The specific update process is as follows:

α为权重系数，i＝1，2，...，m，

M′＝CNN(M)，g＝[P′，M′]，H⁰＝g，即为经特征矩阵加权后和经过CNN编码后的本地地图构建成的一张图；k代表的是在当前在GCN的第l层进行计算，

代表的是可训练的权重矩阵，在训练时进行更新；最后，三者相乘的结果经过激活函数后得到H^k+1；利用图卷积神经网络进行特征传递，能捕捉输入的车辆特征矩阵与加权邻接矩阵的关系特征。where H ^k is the hidden matrix, when k=0,

α is the weight coefficient, i=1, 2,..., m,

M'=CNN(M), g=[P', M'], H ⁰ =g, which is a map constructed from the local map weighted by the feature matrix and coded by CNN; k represents the Currently computing at layer l of GCN,

Represents the trainable weight matrix, which is updated during training; finally, the result of the multiplication of the three is obtained after the activation function to obtain H ^k+1 ; The graph convolutional neural network is used for feature transfer, which can capture the input vehicle feature matrix Relational features with weighted adjacency matrix.

Facing the problem of vehicle driving behavior prediction in the Internet of Vehicles scenario, the present invention proposes a vehicle driving behavior prediction method based on WGCN (Weighted Graph Neural Network). This method first generates a feature matrix, and then constructs a graph with the weighted sum of the local map encoded by CNN, and then uses the edge-enhanced attention mechanism of GCN to increase the dimension of edge features, which improves the accuracy of weight coefficient assignment, making The extracted interactive features are more abundant; secondly, the feature transfer mechanism of the GCN graph convolutional neural network enables the interactive features of vehicles to be transferred in the form of graphs, which fully represent the changes in the interactive relationship between vehicles, so as to predict the driving behavior of vehicles. have higher accuracy. Finally, the Transformer is input for training, and the prediction result is obtained through the fully connected layer, which can better predict the driving behavior of the vehicle.

Description of drawings

1 is a flowchart of a WGCN-based vehicle driving behavior prediction method in an embodiment;

2 is a schematic diagram of a vehicle driving behavior prediction network model in an embodiment;

FIG. 3 is a schematic diagram of an edge-enhanced attention mechanism in an embodiment.

Detailed ways

In order to further understand the content of the present invention, the present invention will be described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and not for limiting.

Example

As shown in FIG. 1 , this embodiment provides a WGCN-based vehicle driving behavior prediction method, which includes the following steps:

1. Generate the feature matrix and local map of each vehicle;

2. The weighted feature matrix and the local map encoded by CNN are constructed into a graph and input into the edge-enhanced graph convolutional neural network GCN;

3. Extract the features of the input data through GCN, and use the edge-enhanced attention mechanism of GCN to increase the dimension of edge features and improve the accuracy of weight coefficient allocation; use the feature transfer mechanism of GCN to make the interactive features of vehicles in the form of graphs. The form is transmitted, which fully expresses the change of the interaction relationship between the vehicles;

4. Input the interactive features output by GCN into Transformer for training;

5. Obtain the prediction result of vehicle driving behavior through the fully connected layer.

In the edge-enhanced attention mechanism, vertex n and the edge feature vector of surrounding vertices calculate the weight value of surrounding vehicles. The ultimate goal is to generate an adjacency matrix with weights to represent the influence between different vehicles. The adjacency matrix represents as follows:

A'=softmax(A)

The above formula describes the process of the attention matrix A'. First, the edge feature matrix E needs to be normalized to obtain E', and then the edge feature matrix is multiplied by the trainable attention parameter matrix W _a to obtain the attention matrix A, Then perform softmax calculation on the matrix A to obtain A', so that the value of the elements in A' is in the range of 0 to 1, so as to express different weights; finally, the weighted adjacency matrix A _adj is obtained:

A _adj =E'A'.

In the feature transfer mechanism, the adjacency relationship and interaction features between vehicles are used as the input of the neural network in the form of a graph, and the feature transfer method of the graph convolutional neural network is used to update the information, so that the network can fully extract the internal relationship between vehicles;

The feature matrix X and the weighted adjacency matrix A _adj are used as update information to propagate in the constructed interaction model. The specific update process is as follows:

α为权重系数，i＝1，2，...，m，

M′＝CNN(M)，g＝[P′，M′]，H⁰＝g，即为经特征矩阵加权后和经过CNN编码后的本地地图构建成的一张图；k代表的是在当前在GCN的第l层进行计算，

代表的是可训练的权重矩阵，在训练时进行更新；最后，三者相乘的结果经过激活函数后得到H^k+1；利用图卷积神经网络进行特征传递，能捕捉输入的车辆特征矩阵与加权邻接矩阵的关系特征。where H ^k is the hidden matrix, when k=0,

α is the weight coefficient, i=1, 2,..., m,

M'=CNN(M), g=[P', M'], H ⁰ =g, which is a map constructed from the local map weighted by the feature matrix and coded by CNN; k represents the Currently computing at layer l of GCN,

Represents the trainable weight matrix, which is updated during training; finally, the result of the multiplication of the three is obtained after the activation function to obtain H ^k+1 ; The graph convolutional neural network is used for feature transfer, which can capture the input vehicle feature matrix Relational features with weighted adjacency matrix.

The overall execution speed of Transformer is faster. For the same task convergence rounds, Transformer converges nearly ten times faster than LSTM, and Transformer training is also faster than LSTM.

Transformer does not use recursion, Transformer achieves infinite attention span by using global contrast. Instead of processing each agent in sequence, it processes the entire sequence at once and creates an attention matrix, where each output is a weighted sum of the inputs. For example, in natural language processing, the French word "accord" can be represented as "The(0)+agreement"(1)+... The neural network learns the weights of the attention matrix.

However, Transformer lacks modeling of the time dimension, and even if there is Position Encoding, it is still far from the natural time series network such as LSTM.

And this problem can be solved by the historical weighted graph. The historical weighted graph contains time information, so Transformer is not needed for modeling.

network model

Figure 2 is a schematic diagram of a network model for predicting the driving behavior of vehicles on a highway, depicting vehicles traveling in multiple lanes. The Cloud Server in the figure is a cloud server located in the cloud, which is the data center and computing center for forecasting tasks, and the MEC server is a server located at the edge that assists the cloud server in storage and computing. The ego vehicle and surrounding vehicles can transmit information through V2V, and the vehicle can also communicate with MEC and CloudServer through V2I. When the ego vehicle sends a prediction request, it needs to obtain the current driving information and historical trajectory data of the surrounding vehicles, and then predict the future driving behavior of the surrounding vehicles according to the obtained data, such as the vehicle keeping the speed and driving straight, the vehicle accelerating the driving, the vehicle Vehicle driving behaviors such as slowing down, turning left, or turning right. In the scenario of this embodiment, a plurality of MEC servers located in different geographical locations are set up, and these servers will perform data collection and deep learning tasks simultaneously with the vehicle. The advantage of this setting is that the computing and storage capacity of each MEC server and the vehicle can be fully utilized, and all the information can be combined to obtain more complete data.

Edge-augmented graph convolutional neural network model

In order to better extract the adjacency and interaction features between vehicles in complex dynamic scenes, this embodiment designs a graph convolutional neural network model based on the edge-enhanced attention mechanism. The model has two important mechanisms, namely edge- Enhanced attention mechanism and feature transfer mechanism based on graph convolutional neural network. The edge-enhanced attention mechanism increases the dimension of edge features, improves the accuracy of weight coefficient allocation, and makes the extracted interactive features more abundant; the feature transfer mechanism of graph convolutional neural network is to introduce node feature matrix and weighted adjacency matrix to The data form of the dynamic graph transmits and updates the node interaction features, so as to fully describe the changes in the interaction relationship between the ego vehicle and the surrounding vehicles.

Edge-Enhanced Attention Mechanism

Edge-enhancement in the attention mechanism refers to increasing the dimension of edge features, so that edge features can express more information; attention mechanism refers to assigning different weight coefficients to different vertices. Vertices have different priorities when they are processed. The higher the weight of the vertex, the richer the information of the vertex, the greater the influence.

In the actual traffic scene, the driving behavior of the vehicle has high complexity, so to obtain a better prediction effect, it is necessary to fully extract the interactive features of the vehicle. Traditional graph neural network models such as Graph Attention Network (GAT) and Graph Convolutional Network (GCN) cannot meet this requirement well, because GCN only considers two Whether there is an edge between vertices, and although GAT can add weight to the edge to indicate the influence of each vertex, the feature of the edge can only be a real number, which means that the features contained in the edge are not rich enough. All in all, these two The traditional graph neural network model cannot fully express the features of edges, so it cannot effectively extract the features of vehicle interaction.

The attention mechanism is reflected in the relative state between vehicles. For example, a vehicle is very close to the ego vehicle and is prone to collision. Therefore, a higher weight is assigned to this vehicle, which makes the impact of this vehicle larger. The improvement of the edge-enhanced attention mechanism is that the feature of the edge can be multi-dimensional, not just a real number, so that the edge contains more information and the weight coefficient assignment is more accurate. As shown in Figure 3, vertex n and the edge eigenvectors of surrounding vertices calculate the weight value of surrounding vehicles. The ultimate purpose of this step is to generate an adjacency matrix with weights to represent the influence between different vehicles. The adjacency matrix The specific representation is as follows:

A'=softmax(A)

The above formula describes the process of the attention matrix A'. First, the edge feature matrix E needs to be normalized to obtain E', and then the edge feature matrix is multiplied by the trainable attention parameter matrix Wa to obtain the attention matrix _A. , and then perform a softmax calculation on the matrix A to obtain A', so that the values of the elements in A' are in the range of 0 to 1, so as to represent different weights. Finally, the weighted adjacency matrix A _adj is obtained:

A _adj =E'A'.

Feature Transfer Mechanism of Graph Convolutional Neural Networks

The adjacency relationship and interaction features between vehicles are used as the input of the neural network in the form of a graph, and the feature transfer method of the graph convolutional neural network is used to update the information, so that the network can more fully extract the internal relationship between the vehicles. From the above, the feature matrix X and the weighted adjacency matrix A _adj can be obtained. These two matrices are used as update information to propagate in the constructed interaction model. The specific update process is as follows:

α为权重系数，i＝1，2，...，m，

M′＝CNN(M)，g＝[P′，M′]，H⁰＝g，即为经特征矩阵加权后和经过CNN编码后的本地地图构建成的一张图。k代表的是在当前在GCN的第k层进行计算，

代表的是可训练的权重矩阵，在训练时进行更新。最后，三者相乘的结果经过激活函数后得到H^k+1。因此，利用图卷积神经网络进行特征传递，可以较好的捕捉输入的车辆特征矩阵以及带权邻接矩阵的关系的特征，实际上就是可以对复杂的交通图进行信息提取，以供后面的步骤处理，综上，GCN的本质就是一个信息提取器。where H ^k is the hidden matrix, when k=0,

α is the weight coefficient, i=1, 2,..., m,

M'=CNN(M), g=[P', M'], H ⁰ =g, which is a map constructed from the local map weighted by the feature matrix and encoded by the CNN. k represents the current calculation in the kth layer of GCN,

Represents a trainable weight matrix, which is updated during training. Finally, the result of the multiplication of the three is passed through the activation function to obtain H ^k+1 . Therefore, the use of graph convolutional neural network for feature transfer can better capture the characteristics of the relationship between the input vehicle feature matrix and the weighted adjacency matrix. In fact, it is possible to extract information from complex traffic maps for later steps. Processing, to sum up, the essence of GCN is an information extractor.

Problem Definition and Modeling

The normal driving of the vehicle on the highway will produce different driving behaviors, such as: changing lanes, going straight, etc. Therefore, this embodiment defines the driving behavior of the vehicle. Vehicle driving behavior refers to the behavior that the vehicle makes corresponding actions on the road according to different traffic conditions, so that the vehicle driving state changes. The types of behavior can be firstly divided into three categories: Keep Lane (KL), Turn Left (TL), Turn Right (TR), which can be recorded as: Act ³ ={KL,TL,TR}. On this basis, driving behaviors can be further divided into: Keep Lane And Accelerate (KLA), Keep Lane And Speed (KLS), Keep Lane And Slow Down (KLD), Turn Left Lane change (Turn Left, TL), right lane change (Turn Right, TR), denoted as Act ⁵ ={KLA, KLS, KLD, TL, TR}.

Therefore, the problem to be solved in this embodiment can be described as: given the node feature matrix X of the ego vehicle X ₀ and the surrounding vehicles X _k (k∈[1,n]), and the historical feature S in the time T _his , for Predict the driving behavior of the vehicle after T _fut time:

Predict:

Y ^pre = {y ₀ , y ₁ , ..., y _n }

Subject to:

Get the center vehicle with lane changing behavior

In general, vehicles tend to drive in the original lane, and lane-changing behaviors are relatively rare. Therefore, when choosing a self-vehicle, it is necessary to select vehicles with lane-changing behaviors, so that the driving behavior of surrounding vehicles can be predicted. It is meaningful, so the data of lane-changing vehicles are filtered out from the original data, and the algorithm is as follows:

Algorithm 1: Lane Change Vehicle Selection

The main function of Algorithm 1 is to select the vehicle data F with lane changing behavior from the I-80 data S of the NGSIM. The key point is that the vehicles with the same Vehicle_ID need to be arranged in chronological order, and the data of the vehicles whose Lane_ID changes are added to F. Subsequent steps will operate based on this data.

Get surrounding vehicles of the center vehicle

Algorithm 2: Select the surrounding vehicles of the center vehicle

The sensors deployed on the vehicle have a certain distance limit, and at the same time, the mutual influence between the vehicle and the vehicle is also limited by the distance. The farther the distance between the vehicles, the smaller the influence, so it is necessary to select the vehicle within a certain distance from the center vehicle as the surrounding vehicle. As shown in Algorithm 2, according to the Vehicle_ID, Frame_ID of the vehicle, determine the center vehicle and all vehicles existing in the current frame, and calculate the distance between each vehicle and the center vehicle, and select the vehicle within the distance dis as the effective surrounding vehicle.

Get edge feature matrix

It can be seen from the above that the vehicle interaction model designed in this embodiment is a graph structure, which has a feature matrix X and an edge feature E. The acquisition of the feature matrix X is relatively simple and can be obtained directly according to the formula, and the edge feature matrix E is relatively complex. The acquisition of matrix E is described. The key step is to process the diagonal elements so that each element of the edge feature matrix E is not 0.

Algorithm 3: Calculate the edge feature matrix

The present invention and its embodiments have been described above schematically, and the description is not restrictive, and what is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Therefore, if those of ordinary skill in the art are inspired by it, without departing from the purpose of the present invention, any structural modes and embodiments similar to this technical solution are designed without creativity, which shall belong to the protection scope of the present invention. .

Claims (5)

1. A WGCN-based vehicle driving behavior prediction method, characterized in that it comprises the following steps: 1. Generate the feature matrix and local map of each vehicle; 2. The weighted feature matrix and the local map encoded by CNN are constructed into a map and input into GCN; 3. Extract the features of the input data through GCN, and use the edge-enhanced attention mechanism of GCN to increase the dimension of edge features and improve the accuracy of weight coefficient allocation; use the feature transfer mechanism of GCN to make the interactive features of vehicles in the form of graphs. The form is transmitted, which fully expresses the change of the interaction relationship between the vehicles; 4. Input the interactive features output by GCN into Transformer for training; 5. Obtain the prediction result of vehicle driving behavior through the fully connected layer. 2. The WGCN-based vehicle driving behavior prediction method according to claim 1, characterized in that: in step 1, a feature matrix X is generated, X=[P, M], wherein P is a node feature matrix, including the position (x , y), speed (v _x , v _y ), and heading angle θ, M is the local map;

3. The WGCN-based vehicle driving behavior prediction method according to claim 1, wherein in step 2, the edge-enhancement in the edge-enhancing attention mechanism is to increase the dimension of the edge feature, so that the edge feature can express more Multi-information; the attention mechanism is to assign different weight coefficients to different vertices, and vertices with different weights have different priorities during processing. The higher the weight of a vertex, the richer the information of the vertex, and the greater the influence. 4. The WGCN-based vehicle driving behavior prediction method according to claim 3, characterized in that: in the edge-enhanced attention mechanism, vertex n and the edge feature vector of surrounding vertices calculate the weight value of surrounding vehicles, and the final purpose is Generate an adjacency matrix with weights to represent the influence between different vehicles. The adjacency matrix is expressed as follows:

A'=softmax(A) The above formula describes the process of the attention matrix A'. First, the edge feature matrix E needs to be normalized to obtain E', and then the edge feature matrix is multiplied by the trainable attention parameter matrix W _a to obtain the attention matrix A, Then perform softmax calculation on the matrix A to obtain A', so that the value of the elements in A' is in the range of 0 to 1, so as to express different weights; finally, the weighted adjacency matrix A _adj is obtained: A _adj =E'A'. 5. The WGCN-based vehicle driving behavior prediction method according to claim 4, characterized in that: in the feature transfer mechanism, the adjacency relationship and interaction features between vehicles are used as the input of the neural network in the form of graphs, and the graph convolutional neural The network's feature transfer method is used to update information, so that the network can fully extract the internal relationship between vehicles; The feature matrix X and the weighted adjacency matrix A _adj are used as update information to propagate in the constructed interaction model. The specific update process is as follows:

where H ^k is the hidden matrix, when k=0,

α is the weight coefficient, i=1, 2,..., m,

M'=CNN(M), g=[P', M'], H ⁰ =g, which is a map constructed from the local map weighted by the feature matrix and coded by CNN; k represents the Currently computing at layer l of GCN,

Represents the trainable weight matrix, which is updated during training; finally, the result of the multiplication of the three is obtained after the activation function to obtain H ^k+1 ; The graph convolutional neural network is used for feature transfer, which can capture the input vehicle feature matrix Relational features with weighted adjacency matrix.

Images