WO2023221348A1 - Vehicle trajectory prediction method and system, computer device and storage medium - Google Patents

Abstract

The present invention relates to the field of automatic driving. Disclosed are a vehicle trajectory prediction method and system, a computer device and a storage medium. The method comprises: extracting historical motion trajectory data of a vehicle to be predicted and adjacent vehicles of the vehicle to be predicted to obtain a multi-dimensional dynamic scene feature vector of the vehicle to be predicted; extracting the multi-dimensional dynamic scene feature vector of the vehicle to be predicted to obtain traffic sensing information of the vehicle to be predicted; encoding the traffic sensing information and historical motion state data of the vehicle to be predicted to obtain hidden state information of the vehicle to be predicted; and obtaining, according to the hidden state information of the vehicle to be predicted, a hybrid attention matrix of the vehicle to be predicted, allocating a weight to the hidden state information of the vehicle to be predicted by means of the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain a trajectory prediction value of the vehicle to be predicted, so that the vehicle trajectory prediction accuracy is effectively improved.

Description

Vehicle trajectory prediction method, system, computer equipment and storage medium Technical field

The invention belongs to the field of automatic driving and relates to a vehicle trajectory prediction method, system, computer equipment and storage medium.

Background technique

Autonomous vehicles and connected cars will play a key role in the future development of road traffic and transportation. In order to be able to travel quickly and safely in complex traffic environments, connected cars must use path planning algorithms and driving strategies based on the future trajectories of surrounding vehicles. Decide when to accelerate, change lanes, etc. However, due to the influence of the driver's subjective driving intention and the objective dynamic interactions between vehicles, vehicle trajectories usually exhibit high nonlinearity. Therefore, predicting the future trajectory of the vehicle is a very challenging problem.

Currently, only the interaction between vehicles is considered when conducting trajectory prediction, and the impact of the driver's subjective driving intention on the vehicle's future trajectory is ignored, resulting in low trajectory prediction accuracy, especially when the vehicle maneuvers laterally. In order to solve this problem, trajectory prediction methods based on driving strategy classification have become a research direction. This type of method first predicts the future driving strategy of the vehicle, such as going straight, changing lanes left, changing lanes right, etc., and then performs microscopic trajectories based on the driving strategy. Prediction. Although this type of method solves some of the lane change trajectory prediction problems, it also makes the trajectory prediction accuracy completely dependent on the prediction accuracy of the driving strategy. When the model predicts the driving strategy incorrectly, the trajectory prediction accuracy drops significantly. In addition, the long-term prediction ability of the LSTM-based model is poor, and its prediction error increases sharply as the prediction time increases. As for the Transformer model, which has strong long-term prediction capabilities, its model parameters and calculations are huge, making the model too complex.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings of low vehicle trajectory prediction accuracy in the above-mentioned prior art and provide a vehicle trajectory prediction method, system, computer equipment and storage medium.

In order to achieve the above objectives, the present invention adopts the following technical solutions to achieve:

The first aspect of the present invention, a vehicle trajectory prediction method, includes:

Obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted;

Through the preset multi-dimensional dynamic scene feature extraction function, the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted are extracted, and the multi-dimensional dynamic scene feature vector of the vehicle to be predicted is obtained;

Extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through the preset information extraction neural network to obtain the traffic perception information of the vehicle to be predicted;

Through the preset time feature encoder, the traffic perception information and historical motion state data of the vehicle to be predicted are encoded to obtain the hidden state information of the vehicle to be predicted;

According to the hidden state information of the vehicle to be predicted, the hybrid attention matrix of the vehicle to be predicted is obtained, and the weight is assigned to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then through maximum pooling and full connection in sequence Process to obtain the trajectory prediction value of the vehicle to be predicted.

Optionally, each of the adjacent vehicles of the vehicle to be predicted includes adjacent vehicles in eight directions: front, rear, left, right, front left, rear left, front right and rear right of the vehicle to be predicted.

Optionally, extracting the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function, and obtaining the multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes:

Through the preset multi-dimensional dynamic scene feature extraction function, extract the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted, the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction. Based on the position data, speed data and acceleration data, the multi-dimensional dynamic scene feature vector of the vehicle to be predicted is obtained.

Optionally, the information extraction neural network includes a first convolution layer, a second convolution layer, a maximum pooling layer and a fully connected layer connected in sequence;

Among them, the first convolution layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction, to obtain the vehicle to be predicted and the vehicle to be predicted. The first fusion feature of adjacent vehicles;

The second convolutional layer is used to fuse the first fusion features of the vehicle to be predicted and any adjacent vehicles among the adjacent vehicles of the vehicle to be predicted, to obtain the second fusion feature;

The maximum pooling layer is used for maximum pooling to process the second fusion feature;

The fully connected layer is used to fully connect the second fusion feature after the maximum pooling process to obtain the traffic perception information of the vehicle to be predicted.

Optionally, the preset temporal feature encoder is a long short-term memory network encoder.

Optionally, obtaining the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted includes:

The hybrid attention matrix α of the vehicle to be predicted is obtained by the following formula:

α _t =softmax(g _t (H,h _t+1 ))

α _f =softmax(g _f (H, h _t+1 ))

α=α _t α _f

Among them, softmax is the normalized exponential function, α _t is the time weight vector, α _f is the feature weight vector, g _t (H, h _t+1 ) = Hh ^T , g _f (H, h _t+1 ) = h _t+ 1(W _f H),

g _f is the feature weight cosine correlation function, g _t is the time weight cosine correlation function, W _f is the feature matrix of the vehicle to be predicted in the historical T _h frame, H is the hidden state information of the vehicle to be predicted in the historical T _h frame, h ^t is the hidden state data of the vehicle to be predicted at time t, h _t+1 is the hidden state data of the vehicle to be predicted at time t+1.

Optionally, assigning weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then sequentially performing maximum pooling processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted includes:

The trajectory prediction value y _t+1 of the vehicle to be predicted is obtained through the following formula:

O＝α⊙H

h′ _t+1 =contact(h _t+1 ,o _t ,o _f )

y _t+1 ＝h′ _t+1 W ₂ W ₁

Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o _t is the most beneficial moment to improve the prediction accuracy,

In order to perform maximum pooling in the time dimension, O _i,j is the hidden state information of the vehicle to be predicted after assigning weights, _of is the most beneficial feature to improve prediction accuracy,

In order to perform maximum pooling processing in the feature dimension, h′ _t+1 is the hidden state information of the vehicle to be predicted at time t+1 obtained by fully connecting o _t , of _f and h _t+1 , and contact is fully connected processing. , W ₁ and W ₂ are preset weights.

The second aspect of the present invention, a vehicle trajectory prediction system, includes:

The data acquisition module is used to obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted;

The data preprocessing module is used to extract the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through the preset multi-dimensional dynamic scene feature extraction function, and obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted;

The information extraction module is used to extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network to obtain the traffic perception information of the vehicle to be predicted;

The encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted;

The prediction module is used to obtain the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then pass through the maximum pool in sequence ization processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted.

A third aspect of the present invention is a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vehicle trajectory is realized. Prediction steps.

A fourth aspect of the present invention is a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of vehicle trajectory prediction are implemented.

Compared with the prior art, the present invention has the following beneficial effects:

The vehicle trajectory prediction method of the present invention uses the historical movement trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted, and performs feature extraction through a preset multi-dimensional dynamic scene feature extraction function to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted. , and based on the multi-dimensional dynamic scene feature vector of the vehicle to be predicted, the traffic perception information of the vehicle to be predicted is obtained through the information extraction neural network, thereby representing the dynamic dependencies between the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted, and the driver's The subjective driving intention is then passed through the temporal feature encoder to encode the traffic perception information and historical motion state data of the vehicle to be predicted to obtain the hidden state information of the vehicle to be predicted, and then based on the hidden state information of the vehicle to be predicted, a mixture of the vehicle to be predicted is obtained attention matrix, and finally allocate weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and selectively reuse the historical trajectory information through the hybrid attention mechanism to improve the long-term prediction ability of the model, and then in turn Through maximum pooling processing and full connection processing, the trajectory prediction value of the vehicle to be predicted is obtained, making the vehicle trajectory prediction more reasonable and accurate.

Description of the drawings

Figure 1 is a flow chart of a vehicle trajectory prediction method according to an embodiment of the present invention;

Figure 2 is a schematic diagram of the traffic scene and static coordinate system according to the embodiment of the present invention;

Figure 3 is a schematic diagram of the construction of a multi-dimensional dynamic scene feature map according to an embodiment of the present invention;

Figure 4 is a schematic structural diagram of an information extraction neural network according to an embodiment of the present invention;

Figure 5 is a schematic diagram of the principle of the hybrid attention mechanism according to an embodiment of the present invention;

Figure 6 is a schematic diagram of the prediction principle based on the hybrid attention mechanism according to the embodiment of the present invention;

Figure 7 is a schematic diagram of the principle of the vehicle trajectory prediction method according to the embodiment of the present invention;

Figure 8 is a data preprocessing flow chart according to the embodiment of the present invention;

Figure 9 is a schematic diagram of data distribution before normalization according to the embodiment of the present invention;

Figure 10 is a schematic diagram of normalized data distribution according to the embodiment of the present invention;

Figure 11 is a schematic diagram of the relationship between the initial learning rate and the number of iterations according to the embodiment of the present invention;

Figure 12 is a trajectory prediction diagram of the vehicle trajectory prediction method of the present invention according to the embodiment of the present invention;

Figure 13 is a trajectory prediction diagram of the existing transformer model according to the embodiment of the present invention;

Figure 14 is a schematic diagram of the relationship between the RMSE of a sample trajectory and the number of predicted frames according to an embodiment of the present invention;

Figure 15 is a schematic diagram of the relationship between the MAEx of a sample trajectory and the number of predicted frames according to the embodiment of the present invention;

Figure 16 is a schematic diagram of the relationship between MAEy and the number of predicted frames for a sample trajectory according to an embodiment of the present invention;

Figure 17 is a schematic diagram of the relationship between the RMSE of another sample trajectory and the number of predicted frames according to an embodiment of the present invention;

Figure 18 is a schematic diagram of the relationship between MAEx and the number of predicted frames of another sample trajectory according to an embodiment of the present invention;

Figure 19 is a schematic diagram of the relationship between MAEy and the number of predicted frames of another sample trajectory according to an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein are capable of being practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

The present invention will be described in further detail below in conjunction with the accompanying drawings:

Referring to Figure 1, one embodiment of the present invention provides a vehicle trajectory prediction method, which solves the problem of low prediction accuracy and poor long-term prediction ability due to existing methods that do not consider the driver's subjective driving intention. Specifically, the vehicle trajectory prediction method includes the following steps:

S1: Obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted.

S2: Through the preset multi-dimensional dynamic scene feature extraction function, extract the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted, and obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

S3: Extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through the preset information extraction neural network, and obtain the traffic perception information of the vehicle to be predicted.

S4: Use the preset temporal feature encoder to encode the traffic perception information and historical motion state data of the vehicle to be predicted, and obtain the hidden state information of the vehicle to be predicted.

S5: According to the hidden state information of the vehicle to be predicted, obtain the hybrid attention matrix of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then proceed through maximum pooling and Fully connected processing to obtain the trajectory prediction value of the vehicle to be predicted.

Specifically, the vehicle trajectory prediction method of the present invention uses the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted, and performs feature extraction through a preset multi-dimensional dynamic scene feature extraction function to obtain the multi-dimensional dynamics of the vehicle to be predicted. Scene feature vector, and based on the multi-dimensional dynamic scene feature vector of the vehicle to be predicted, the traffic perception information of the vehicle to be predicted is obtained through the information extraction neural network, and the dynamic dependence between the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted is represented. and the driver's subjective driving intention, and then use the temporal feature encoder to encode the traffic perception information and historical motion state data of the vehicle to be predicted to obtain the hidden state information of the vehicle to be predicted, and then based on the hidden state information of the vehicle to be predicted, the vehicle to be predicted is obtained The hybrid attention matrix of the vehicle finally assigns weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted. The historical trajectory information is selectively reused through the hybrid attention mechanism to improve the long-term prediction ability of the model. , and then through maximum pooling processing and full connection processing in sequence, the trajectory prediction value of the vehicle to be predicted is obtained, making the vehicle trajectory prediction more reasonable and accurate.

First, refer to Figure 2, which shows a feasible traffic scene and static coordinate system during the specific implementation of the vehicle trajectory prediction method of the present invention. The traffic scene is a two-way eight-lane road, and a fixed reference frame is used to determine the position of each vehicle. The starting points of both the x-axis and the y-axis are the upper left corner of the road. The x-axis is parallel to the direction of vehicle movement on the highway, and the y-axis is perpendicular to the direction of travel. Along the y-axis direction, there are 1-8 lanes. Among them, vehicles in lanes 1-4 travel along the positive direction of the x-axis, and vehicles in lanes 5-8 travel in the negative direction of the x-axis. In the following embodiments, a traffic scene is taken as an example for schematic description, but the sequence is not limited.

In a possible implementation, each of the adjacent vehicles of the vehicle to be predicted includes adjacent vehicles in eight directions: front, rear, left, right, front left, rear left, front right and rear right of the vehicle to be predicted.

Optionally, extracting the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted through a preset multi-dimensional dynamic scene feature extraction function, and obtaining the multi-dimensional dynamic scene feature vector of the vehicle to be predicted includes: using the preset The multi-dimensional dynamic scene feature extraction function extracts the position of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction from the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted. Data, speed data and acceleration data are used to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

Specifically, see Figure 3. The multi-dimensional dynamic scene feature vector of the vehicle to be predicted is extracted through the preset information extraction neural network. The specific extraction process can be completed by constructing a multi-dimensional dynamic scene feature map. Among them, the multi-dimensional dynamic scene feature map is constructed with the vehicle to be predicted as the center and the vehicles in eight directions around it, namely the front, rear, left, right, left front, left rear, right front and right rear of the predicted vehicle. The 3*3 rectangular area is divided into 9 units, and adjacent vehicles are mapped to corresponding units according to their relative positions. According to this rule, the position layer, velocity layer and acceleration layer in the X and Y directions are constructed respectively, forming a total of 6 rectangles with a dimension of 3*3, and finally superimposed to form a multi-dimensional dynamic scene feature map with a dimension of 6*3*3 . Assume that the definition function of the feature map is

Then the multi-dimensional dynamic scene characteristics at time t can be expressed as:

Among them, x _t represents the motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted at time t.

In a possible implementation, the information extraction neural network includes a first convolution layer, a second convolution layer, a maximum pooling layer and a fully connected layer connected in sequence.

Among them, the first convolution layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction, to obtain the vehicle to be predicted and the vehicle to be predicted. The first fusion feature of adjacent vehicles; the second convolutional layer is used to fuse the first fusion feature of the vehicle to be predicted and any adjacent vehicle among the adjacent vehicles of the vehicle to be predicted to obtain the second fusion feature; the maximum pooling layer is used The second fusion feature is processed by maximum pooling; the fully connected layer is used to fully connect the second fused feature after maximum pooling to obtain the traffic perception information of the vehicle to be predicted.

Specifically, see Figure 4. For the multi-dimensional dynamic scene volume of 6*3*3, the first convolutional layer of 6*1*1 is used to perform feature fusion on the 6-dimensional features of each vehicle, which can capture the driving behavior of each vehicle. member’s intentions. Then, each vehicle interacts with surrounding vehicles through a second convolutional layer of 2*2 to extract the interaction between vehicles. Finally, the traffic force constraint information p that reflects the driver's driving intention and the interaction between vehicles is obtained through the maximum pooling layer Maxpool and the fully connected layer FC. The above process is defined as a convolution operation in which the mapping function is traf, and the traffic perception information p ^t is extracted from the multi-dimensional dynamic scene feature vector F ^t at time t through the traf function:

p ^t =traf(F ^t ).

In a possible implementation, the preset temporal feature encoder is a long short-term memory network (LSTM) encoder. Specifically, when the traffic sensing information and historical motion state data of the vehicle to be predicted are encoded through the preset temporal feature encoder to obtain the hidden state information of the vehicle to be predicted, the traffic sensing information p ^t and the historical motion of the vehicle to be predicted are trajectory data

Splicing, as the input of the LSTM encoder, encodes the historical state through the following formula:

f ^t =σ(W _fz zt+W _fh h ^t-1 +b _f )

i ^t =σ(W _iz z ^t +W _ih h ^t-1 +b _i )

o ^t =σ(W _oz z ^t +W _oh h ^t-1 +b _o )

Among them, the forgetting gate f ^t in the LSTM encoder determines what information is discarded, the input gate i ^t controls what new information is updated and stored, and the output gate o ^t controls the output of the candidate layer, the value

is the candidate unit, and the cell state c ^t of the LSTM encoder is the cell state c ^t-1 at the previous moment and the current candidate state

The combination of , the final output gate ^o of the LSTM encoder will determine which cell states are output.

Then the hidden state information of the vehicle to be predicted is obtained through the following formula:

h ^t ＝o ^t ⊙tanh(c ^t )

Among them, σ is the sigmoid function, ⊙ is the multiplication of the corresponding elements of the matrix, x ^t and h ^t are the input vector and hidden layer state at time t respectively. f, i and o are the forgetting gate, input gate and output gate respectively, W and b are model parameters, W _fz is the forgetting gate input weight matrix, W _fh is the forgetting gate hidden layer weight matrix, W _iz is the input gate input weight matrix , W _ih is the input gate hidden layer weight matrix, W _oz is the output gate input weight matrix, W _oh is the output gate hidden layer weight matrix, W _cz is the cell state input weight matrix, W _ch is the cell state hidden layer weight matrix, b _f is the forget gate bias, b _i is the input gate bias, b _o is the output gate bias, and b _c is the cell state bias.

In a possible implementation, obtaining the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted includes: obtaining the hybrid attention matrix α of the vehicle to be predicted by the following formula:

α _t =softmax(g _t (H,h _t+1 ))

α _f =softmax(g _f (H, h _t+1 ))

α=α _t α _f

Among them, softmax is the normalized exponential function, α _t is the time weight vector, α _f is the feature weight vector, g _t (H, h _t+1 ) = Hh ^T , g _f (H, h _t+1 ) = h _t+1 (W _f H),

g _f is the feature weight cosine correlation function, g _t is the time weight cosine correlation function, W _f is the feature matrix of the vehicle to be predicted in the historical T _h frame, H is the hidden state information of the vehicle to be predicted in the historical T _h frame, h ^t is the hidden state data of the vehicle to be predicted at time t, h _t+1 is the hidden state data of the vehicle to be predicted at time t+1.

Specifically, see Figure 5. The vehicle trajectory prediction method of the present invention fuses temporal attention and feature attention. The hybrid attention mechanism can comprehensively consider the moments and features that have a great impact on the output accuracy, and provide a prediction for each moment at each moment. A feature independently assigns attention weights. H is the hidden state information of the vehicle to be predicted in the historical _Th frame, that is, the hidden state of the LSTM encoder. If the hidden state of the LSTM encoder is set to n dimensions, then

Assuming that the hidden state at time t+1 is h _t+1 ∈R ^1×n , then the time-weighted cosine correlation function g _t and the feature weight cosine correlation function g _f are used to calculate the relationship between H and h _t+1 at time and feature Dimensional correlation.

Then convert the cosine correlation into a time weight vector through the Softmax function

with the feature weight vector α _f ∈R ^1×n , and finally obtain the hybrid attention matrix through vector multiplication

In a possible implementation, the hidden state information of the vehicle to be predicted is assigned a weight through the hybrid attention matrix of the vehicle to be predicted, and then the trajectory prediction of the vehicle to be predicted is obtained through maximum pooling processing and full connection processing in sequence. The values include: The trajectory prediction value y _t+1 of the vehicle to be predicted is obtained through the following formula:

O＝α⊙H

h′ _t+1 =contact(h _t+1 ,o _t ,o _f )

y _t+1 ＝h′ _t+1 W ₂ W ₁

Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o _t is the most beneficial moment to improve the prediction accuracy,

In order to perform maximum pooling in the time dimension, O _i,j is the hidden state information of the vehicle to be predicted after assigning weights, _of is the most beneficial feature to improve prediction accuracy,

In order to perform maximum pooling processing in the feature dimension, h′ _t+1 is the hidden state information of the vehicle to be predicted at time t+1 obtained by fully connecting o _t , of _f and h _t+1 , and contact is fully connected processing. , W ₁ and W ₂ are the preset weights of the two fully connected layers respectively.

Specifically, see Figure 6. First, use the mixed attention matrix α to assign weights to H; then perform maximum pooling in the time dimension and feature dimension, and get

and of _f ∈R ^1×n ; finally, h _t+1 , o _t and _of are connected, and the trajectory prediction value y _{t+1 at time t+1} is obtained through the fully connected layer.

Referring to Figure 7, the implementation principle of the vehicle trajectory prediction method of the present invention integrates the traffic force-aware LSTM encoder and the LSTM decoder based on the hybrid attention mechanism. x _pre is the historical trajectory data of the vehicle to be predicted, z _t and h _t respectively represents the vector splicing result and hidden state of LSTM at time t. Finally, the LSTM decoder gives the prediction result y t+1 at time _t+1 .

Optionally, the vehicle trajectory prediction method of the present invention includes the preset multi-dimensional dynamic scene feature extraction function, the preset information extraction neural network, the preset temporal feature encoder, and the maximum pooling and full connection processing. The network layers are pre-trained to determine their specific parameters. Among them, during the pre-training process, the trajectory data of the vehicle in the past 50 frames (the vehicle's position data, speed data and acceleration data in the x and y directions) are used to predict the trajectory of the next 50 frames, so a total of 100 frames of complete data are needed. Through data cleaning, vehicle data that appears for less than 100 frames will be deleted.

In order to verify the effectiveness of the vehicle trajectory prediction method of the present invention, the German high-speed public data set highD was used for testing, and 110,660 trajectories were selected for model training and 50,787 trajectories were used for model testing.

The preprocessing process of the data before the experiment is shown in Figure 8. Figures 9 and 10 show the data distribution before and after normalization in the data preprocessing. Figure 11 shows the different initial learning rates used in different training stages during the model training process. Figures 12 and 13 show the prediction results of 50 consecutive frames between the vehicle trajectory prediction method of the present invention and the existing Transformer model. Figures 14 to Figures 19 is the comparison of various evaluation indicators between the vehicle trajectory prediction method of the present invention and the existing Transformer model. The specific instructions are as follows:

See Figure 11, which shows the initial learning rate used by the model in different training stages. In this way, the fluctuation of the model performance can be reduced when it is close to the optimal, and the optimal point can be reached faster.

Referring to Figures 12 and 13, a comparison chart of trajectory prediction between the vehicle trajectory prediction method of the present invention and the existing Transformer model is shown. As can be seen from Figure 12, the prediction results of this method are more accurate when the vehicle maneuvers laterally, while the Transformer model tends to offset the left and right maneuvers in the lateral direction; as can be seen from Figure 13, as the prediction length increases, , the predicted points of the Transformer model are further and further away from the real trajectory points, but the vehicle trajectory prediction method of the present invention can still make more accurate predictions. Among them, hist represents the historical trajectory coordinates, gt represents the future real trajectory coordinates, ours represents the vehicle trajectory prediction method of the present invention, tf represents the Transformer model, the horizontal coordinate is the vehicle x-direction coordinate, and the vertical coordinate is the vehicle y-direction coordinate.

Referring to Figures 14 to 19, the changes of the evaluation indicators RMSE, MAEx and MAEy with the frame length are shown, where RMSE, MAEx and MAEy respectively represent the root mean square error, the mean absolute error in the x direction and the mean absolute error in the y direction, The abscissa frame represents the time frame. Among them, Figure 14, Figure 15 and Figure 16 respectively show the changes of RMSE, MAEx and MAEy of a sample trajectory with the frame length. Figure 17, Figure 18 and Figure 19 respectively show the changes of RMSE, MAEx and MAEy with the frame length of another sample trajectory. It can be seen that as the prediction frame length increases, the RMSE of the Transformer model increases almost exponentially, and MAEx and MAEy also increase linearly. However, the RMSE of the vehicle trajectory prediction method of the present invention can maintain a low level fluctuation. This shows that the long-term prediction ability of the vehicle trajectory prediction method of the present invention is strong; at the same time, the MAEx of the vehicle trajectory prediction method of the present invention is smaller than Transformer, which also shows that the prediction results of the vehicle trajectory prediction method of the present invention are more accurate when the vehicle is maneuvering sideways.

The following are device embodiments of the present invention, which can be used to perform method embodiments of the present invention. For details not disclosed in the device embodiment, please refer to the method embodiment of the present invention.

In yet another embodiment of the present invention, a vehicle trajectory prediction system is provided, which can be used to implement the above vehicle trajectory prediction method. Specifically, the vehicle trajectory prediction system includes a data acquisition module, a data preprocessing module, an information extraction module, and a coding module. modules as well as prediction modules.

Among them, the data acquisition module is used to obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted; the data preprocessing module is used to extract the vehicle to be predicted and the vehicles to be predicted through a preset multi-dimensional dynamic scene feature extraction function The historical motion trajectory data of each adjacent vehicle is used to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted; the information extraction module is used to extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through the preset information extraction neural network to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted. Traffic perception information; the encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted; the prediction module is used to obtain the hidden state information of the vehicle to be predicted; State information, obtain the hybrid attention matrix of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then sequentially perform maximum pooling processing and full connection processing to obtain the vehicle to be predicted trajectory prediction value.

In a possible implementation, each of the adjacent vehicles of the vehicle to be predicted includes adjacent vehicles in eight directions: front, rear, left, right, front left, rear left, front right and rear right of the vehicle to be predicted.

In a possible implementation, the historical motion trajectory data of the vehicle to be predicted and each adjacent vehicle of the vehicle to be predicted is extracted through a preset multi-dimensional dynamic scene feature extraction function, and a multi-dimensional dynamic scene feature vector of the vehicle to be predicted is obtained. Including: extracting the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through the preset multi-dimensional dynamic scene feature extraction function, and the driving direction and driving direction of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted. The position data, speed data and acceleration data in the vertical direction are used to obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted.

In a possible implementation, the information extraction neural network includes a first convolution layer, a second convolution layer, a maximum pooling layer and a fully connected layer connected in sequence; wherein the first convolution layer is used for fusion The position data, speed data and acceleration data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the traveling direction and the vertical direction of the traveling direction are used to obtain the first fusion characteristics of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted; The two convolutional layers are used to fuse the first fusion features of the vehicle to be predicted and any adjacent vehicles among the adjacent vehicles of the vehicle to be predicted to obtain the second fusion feature; the maximum pooling layer is used to process the second fusion feature by maximum pooling; The fully connected layer is used to fully connect the second fusion feature after the maximum pooling process to obtain the traffic perception information of the vehicle to be predicted.

In a possible implementation, the preset temporal feature encoder is a long short-term memory network encoder.

In a possible implementation, obtaining the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted includes: obtaining the hybrid attention matrix α of the vehicle to be predicted by the following formula:

α _t =softmax(g _t (H,h _t+1 ))

α _f =softmax(g _f (H, h _t+1 ))

α=α _t α _f

Among them, softmax is the normalized exponential function, α _t is the time weight vector, α _f is the feature weight vector, g _t (H, h _t+1 ) = Hh ^T , g _f (H, h _t+1 ) = h _t+1 (W _f H),

g _f is the feature weight cosine correlation function, g _t is the time weight cosine correlation function, W _f is the feature matrix of the vehicle to be predicted in the historical T _h frame, H is the hidden state information of the vehicle to be predicted in the historical T _h frame, h ^t is the hidden state data of the vehicle to be predicted at time t, h _t+1 is the hidden state data of the vehicle to be predicted at time t+1.

In a possible implementation, the hidden state information of the vehicle to be predicted is assigned a weight through the hybrid attention matrix of the vehicle to be predicted, and then the trajectory prediction of the vehicle to be predicted is obtained through maximum pooling processing and full connection processing in sequence. The values include: The trajectory prediction value y _t+1 of the vehicle to be predicted is obtained through the following formula:

O＝α⊙H

h′ _t+1 =contact(h _t+1 ,o _t ,o _f )

y _t+1 ＝h′ _t+1 W ₂ W ₁

Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o _t is the most beneficial moment to improve the prediction accuracy,

In order to perform maximum pooling in the time dimension, O _i,j is the hidden state information of the vehicle to be predicted after assigning weights, _of is the most beneficial feature to improve prediction accuracy,

In order to perform maximum pooling processing in the feature dimension, h′ _t+1 is the hidden state information of the vehicle to be predicted at time t+1 obtained by fully connecting o _t , of _f and h _t+1 , and contact is fully connected processing. , W ₁ and W ₂ are preset weights.

All relevant contents of each step involved in the foregoing embodiments of the vehicle trajectory prediction method can be referred to the functional description of the corresponding functional modules of the vehicle trajectory prediction system in the embodiments of the present invention, and will not be described again here. The division of modules in the embodiments of the present invention is schematic and is only a logical function division. In actual implementation, there may be other division methods. In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit. In the device, it can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules.

In yet another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program. The computer program includes program instructions. The processor is used to execute the computer program. A storage medium stores program instructions. The processor may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or off-the-shelf programmable gate array (Field-Programmable GateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, specifically suitable for Load and execute one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used to operate the vehicle trajectory prediction method.

In yet another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. . It can be understood that the computer-readable storage medium here may include a built-in storage medium in the computer device, and of course may also include an extended storage medium supported by the computer device. The computer-readable storage medium provides storage space, and the storage space stores the operating system of the terminal. Furthermore, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the vehicle trajectory prediction method in the above embodiments.

Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for implementing the functions specified in one process or processes of the flowchart and/or one block or blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that the present invention can still be modified. Modifications or equivalent substitutions may be made to the specific embodiments, and any modifications or equivalent substitutions that do not depart from the spirit and scope of the invention shall be covered by the scope of the claims of the invention.

Claims (10)

1. A vehicle trajectory prediction method, characterized by including:

   Obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted;

   Through the preset multi-dimensional dynamic scene feature extraction function, the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted are extracted, and the multi-dimensional dynamic scene feature vector of the vehicle to be predicted is obtained;

   Extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through the preset information extraction neural network to obtain the traffic perception information of the vehicle to be predicted;

   Through the preset time feature encoder, the traffic perception information and historical motion state data of the vehicle to be predicted are encoded, and the hidden state information of the vehicle to be predicted is obtained;

   According to the hidden state information of the vehicle to be predicted, the hybrid attention matrix of the vehicle to be predicted is obtained, and the weight is assigned to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then through maximum pooling and full connection in sequence Process to obtain the trajectory prediction value of the vehicle to be predicted.
2. The vehicle trajectory prediction method according to claim 1, characterized in that each adjacent vehicle of the vehicle to be predicted includes eight directions of front, rear, left, right, left front, left rear, right front and right rear of the vehicle to be predicted. of adjacent vehicles.
3. The vehicle trajectory prediction method according to claim 1, characterized in that the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted are extracted through a preset multi-dimensional dynamic scene feature extraction function to obtain the vehicle trajectory to be predicted. The multi-dimensional dynamic scene feature vector of the vehicle includes:

   Through the preset multi-dimensional dynamic scene feature extraction function, extract the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted, the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction. Based on the position data, speed data and acceleration data, the multi-dimensional dynamic scene feature vector of the vehicle to be predicted is obtained.
4. The vehicle trajectory prediction method according to claim 3, wherein the information extraction neural network includes a first convolution layer, a second convolution layer, a maximum pooling layer and a fully connected layer connected in sequence;

   Among them, the first convolution layer is used to fuse the position data, speed data and acceleration data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted in the driving direction and the vertical direction of the driving direction, to obtain the vehicle to be predicted and the vehicle to be predicted. The first fusion feature of adjacent vehicles;

   The second convolutional layer is used to fuse the first fusion features of the vehicle to be predicted and any adjacent vehicles among the adjacent vehicles of the vehicle to be predicted, to obtain the second fusion feature;

   The maximum pooling layer is used for maximum pooling to process the second fusion feature;

   The fully connected layer is used to fully connect the second fusion feature after the maximum pooling process to obtain the traffic perception information of the vehicle to be predicted.
5. The vehicle trajectory prediction method according to claim 1, wherein the preset temporal feature encoder is a long short-term memory network encoder.
6. The vehicle trajectory prediction method according to claim 1, wherein obtaining the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted includes:

   The hybrid attention matrix α of the vehicle to be predicted is obtained by the following formula:

   α _t =softmax(g _t (H,h _t+1 ))

   α _f =softmax(g _f (H, h _t+1 ))

   α=α _t α _f

   Among them, softmax is the normalized exponential function, α _t is the time weight vector, α _f is the feature weight vector, g _t (H, h _t+1 ) = Hh ^T , g _f (H, h _t+1 ) = h _t+1 (W _f H),

   

   g _f is the feature weight cosine correlation function, g _t is the time weight cosine correlation function, W _f is the feature matrix of the vehicle to be predicted in the historical T _h frame, H is the hidden state information of the vehicle to be predicted in the historical T _h frame, h ^t is the hidden state data of the vehicle to be predicted at time t, h _t+1 is the hidden state data of the vehicle to be predicted at time t+1.
7. The vehicle trajectory prediction method according to claim 6, characterized in that the hidden state information of the vehicle to be predicted is assigned a weight through the hybrid attention matrix of the vehicle to be predicted, and then sequentially through maximum pooling processing and full connection processing, Obtaining the trajectory prediction value of the vehicle to be predicted includes:

   The trajectory prediction value y _t+1 of the vehicle to be predicted is obtained through the following formula:

   O＝α⊙H

   

   

   h′ _t+1 =contact(h _t+1 ,o _t ,o _f )

   y _t+1 ＝h′ _t+1 W ₂ W ₁

   Among them, ⊙ is the multiplication of the corresponding elements of the matrix, o _t is the most beneficial moment to improve the prediction accuracy,

   

   In order to perform maximum pooling in the time dimension, O _i,j is the hidden state information of the vehicle to be predicted after assigning weights, _of is the most beneficial feature to improve prediction accuracy,

   

   In order to perform maximum pooling processing in the feature dimension, h′ _t+1 is the hidden state information of the vehicle to be predicted at time t+1 obtained by fully connecting o _t , of _f and h _t+1 , and contact is fully connected processing. , W ₁ and W ₂ are preset weights.
8. A vehicle trajectory prediction system, which is characterized by including:

   The data acquisition module is used to obtain the historical movement trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted;

   The data preprocessing module is used to extract the historical motion trajectory data of the vehicle to be predicted and the adjacent vehicles of the vehicle to be predicted through the preset multi-dimensional dynamic scene feature extraction function, and obtain the multi-dimensional dynamic scene feature vector of the vehicle to be predicted;

   The information extraction module is used to extract the multi-dimensional dynamic scene feature vector of the vehicle to be predicted through a preset information extraction neural network to obtain the traffic perception information of the vehicle to be predicted;

   The encoding module is used to encode the traffic perception information and historical motion state data of the vehicle to be predicted through a preset time feature encoder to obtain the hidden state information of the vehicle to be predicted;

   The prediction module is used to obtain the hybrid attention matrix of the vehicle to be predicted based on the hidden state information of the vehicle to be predicted, and assign weights to the hidden state information of the vehicle to be predicted through the hybrid attention matrix of the vehicle to be predicted, and then pass through the maximum pool in sequence ization processing and full connection processing to obtain the trajectory prediction value of the vehicle to be predicted.
9. A computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, it implements claims 1 to 1 7. The steps of vehicle trajectory prediction described in any one of 7.
10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the steps of vehicle trajectory prediction as described in any one of claims 1 to 7 are implemented. .

Images