WO2022222095A1 - Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage - Google Patents

Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage Download PDF

Info

Publication number
WO2022222095A1
WO2022222095A1 PCT/CN2021/088937 CN2021088937W WO2022222095A1 WO 2022222095 A1 WO2022222095 A1 WO 2022222095A1 CN 2021088937 W CN2021088937 W CN 2021088937W WO 2022222095 A1 WO2022222095 A1 WO 2022222095A1
Authority
WO
WIPO (PCT)
Prior art keywords
matrix
trajectory
obstacle
predicted
target
Prior art date
Application number
PCT/CN2021/088937
Other languages
English (en)
Chinese (zh)
Inventor
许家妙
Original Assignee
深圳元戎启行科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳元戎启行科技有限公司 filed Critical 深圳元戎启行科技有限公司
Priority to PCT/CN2021/088937 priority Critical patent/WO2022222095A1/fr
Priority to CN202180050157.3A priority patent/CN115917559A/zh
Publication of WO2022222095A1 publication Critical patent/WO2022222095A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology

Definitions

  • the present application relates to a trajectory prediction method, apparatus, computer equipment, storage medium and vehicle.
  • the autonomous vehicle In the process of autonomous driving, it is very necessary to predict the trajectory of obstacles in the surrounding environment within a certain period of time. By predicting the future trajectory of the obstacle, the autonomous vehicle can identify the intention of the obstacle earlier, and plan the driving route and driving speed according to the intention of the obstacle, so as to avoid collision and reduce the occurrence of safety accidents.
  • the traditional method is to extract features from the historical trajectory information and map information of obstacles through the existing trajectory prediction model to realize trajectory prediction.
  • the network handles raster images or vectorized information.
  • the existing trajectory prediction models can only roughly consider the correlation between the map information and the obstacle trajectory information, and cannot fully extract the deeper level between the map information and the obstacle information. , resulting in a low accuracy of trajectory prediction.
  • a trajectory prediction method comprising:
  • the mechanism Inputting the first trajectory matrix and the first map matrix into the trained trajectory prediction model, and embedding the first trajectory matrix and the first map matrix to obtain a target matrix, based on multi-head attention
  • the mechanism performs feature extraction on the target matrix to obtain output features, and performs regression processing on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • a trajectory prediction device comprising:
  • the trajectory acquisition module is used to acquire the motion trajectory of the obstacle to be predicted
  • a map acquisition module configured to determine target map information corresponding to the to-be-predicted obstacle according to the motion trajectory
  • a matrix conversion module for converting the motion trajectory into a corresponding first trajectory matrix, and converting the target map information into a corresponding first map matrix
  • a trajectory prediction module configured to input the first trajectory matrix and the first map matrix into the trained trajectory prediction model, and perform embedding processing on the first trajectory matrix and the first map matrix to obtain a target Matrix, feature extraction is performed on the target matrix based on the multi-head attention mechanism to obtain output features, and regression processing is performed on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • a computer device comprising a memory and one or more processors, the memory having computer-readable instructions stored therein, the computer-readable instructions, when executed by the processor, cause the one or more processors to execute The following steps:
  • the mechanism Inputting the first trajectory matrix and the first map matrix into the trained trajectory prediction model, and embedding the first trajectory matrix and the first map matrix to obtain a target matrix, based on multi-head attention
  • the mechanism performs feature extraction on the target matrix to obtain output features, and performs regression processing on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • One or more computer storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:
  • the mechanism Inputting the first trajectory matrix and the first map matrix into the trained trajectory prediction model, and embedding the first trajectory matrix and the first map matrix to obtain a target matrix, based on multi-head attention
  • the mechanism performs feature extraction on the target matrix to obtain output features, and performs regression processing on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • a vehicle comprising the steps of executing the above trajectory prediction method.
  • FIG. 1 is an application environment diagram of the trajectory prediction method in one or more embodiments.
  • FIG. 2 is a schematic flowchart of a trajectory prediction method in one or more embodiments.
  • FIG. 3 is a schematic structural diagram of a trained trajectory prediction model in one or more embodiments.
  • FIG. 4 is a schematic flowchart of a step of embedding a first trajectory matrix and a first map matrix to obtain a target matrix in one or more embodiments.
  • FIG. 5 is a block diagram of a trajectory prediction apparatus in one or more embodiments.
  • FIG. 6 is a block diagram of a computer device in one or more embodiments.
  • the trajectory prediction method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the onboard sensor 102 communicates with the onboard computer device 104 over a network.
  • the number of in-vehicle sensors can be one or more.
  • the in-vehicle computer equipment may be simply referred to as computer equipment.
  • the vehicle-mounted sensor 102 sends the collected drive test data to the computer device 104, and the computer device 104 performs detection, tracking and sampling processing on the drive test data to obtain the motion trajectory of the obstacle to be predicted in the road test data, and determines the to-be-predicted obstacle according to the motion trajectory
  • the target map information corresponding to the obstacle so as to convert the motion trajectory into the corresponding first trajectory matrix, and convert the target map information into the corresponding first map matrix, and then input the first trajectory matrix and the first map matrix into the trained
  • the first trajectory matrix and the first map matrix are embedded to obtain the target matrix, and the feature extraction is performed on the target matrix based on the multi-head attention mechanism to obtain the output features, and the output features are subjected to regression processing to obtain the target matrix.
  • the vehicle-mounted sensor 102 can be, but is not limited to, a lidar, a laser scanner.
  • a trajectory prediction method is provided, and the method is applied to the computer device in FIG. 1 as an example for description, including the following steps:
  • Step 202 acquiring the motion trajectory of the obstacle to be predicted.
  • the obstacles to be predicted refer to the dynamic obstacles around the vehicle during the driving process of the vehicle.
  • the obstacles to be predicted may include pedestrians, vehicles, and the like.
  • the sensors installed on the vehicle can send the collected road test data to the computer equipment.
  • the computer equipment can store the drive test data in units of frames, and record the data collection time and other information of each frame of the drive test data.
  • the vehicle sensor can be a lidar, a laser scanner, a camera, and the like.
  • the drive test data can be point cloud data or surrounding environment images.
  • the sensor is a lidar or a laser scanner
  • the collected point cloud data is sent to a computer device.
  • the sensor is a camera
  • the captured image of the surrounding environment is sent to the computer device.
  • the point cloud data refers to the data that the sensor records the scanned surrounding environment information in the form of a point cloud.
  • the surrounding environment information includes the obstacles to be predicted in the surrounding environment of the vehicle, and there can be multiple obstacles to be predicted.
  • the point cloud data may specifically include three-dimensional coordinates of each point, laser reflection intensity, color information, and the like. The three-dimensional coordinates are used to represent the position information of the obstacle surface to be predicted in the surrounding environment.
  • the surrounding environment image may be a panoramic image around the vehicle collected by a plurality of cameras.
  • the computer device Each time the computer device acquires drive test data within a preset time period, it performs target detection and target tracking on the drive test data to obtain a motion trajectory within the preset time period.
  • the preset time period may be 2s.
  • Object detection refers to detecting obstacles in drive test data and predicting the location and category of each obstacle.
  • Target tracking refers to predicting the position of the obstacle in the subsequent frame and determining the speed information of the obstacle to be predicted when the position of the obstacle in the initial frame is known.
  • the track information includes the position information, speed, orientation, etc. of the obstacle to be predicted in each frame of drive test data.
  • the location information refers to the location coordinates of the obstacles to be predicted in the world coordinates.
  • the computer equipment inputs the collected drive test data into the corresponding target detection model, locates the location area where each obstacle to be predicted is located, and uses a bounding box to frame the location area to obtain the corresponding object detection model for each obstacle to be predicted.
  • the bounding box includes the center point coordinates, size, orientation, etc. of each obstacle to be predicted.
  • the coordinates of the center point of the bounding box represent the position information of the obstacle to be predicted.
  • the computer device can input the bounding box of the current frame corresponding to the obstacle to be predicted and a continuous multi-frame bounding box composed of bounding boxes before the current frame into the pre-trained target tracking model to obtain the speed and acceleration of the obstacle to be predicted in the current frame.
  • the computer equipment obtains the trajectory information of the obstacle to be predicted in each frame by performing target detection and target tracking on each frame of drive test data.
  • a high-precision map is stored in the computer equipment, and the high-precision map contains rich and detailed road traffic information elements. High-precision maps not only have high-precision coordinates, but also include accurate road shapes, and also include data on the slope, curvature, heading, elevation, roll, etc. of each lane.
  • a high-resolution map will not only describe the road, but also how many lanes there are on a road, and will truly reflect the actual style of the road.
  • the computer device can sample and process the trajectory information of the obstacle to be predicted based on the high-precision map, and obtain a trajectory that satisfies the preset sampling conditions, thereby obtaining the motion trajectory of the obstacle to be predicted.
  • the preset sampling conditions refer to trajectories in the junction area, trajectories with changes in curvature and speed, and trajectories with lane changes and cut ins.
  • the motion trajectory includes a plurality of trajectory points, and each trajectory point includes coordinate values in the x-direction and the y-direction.
  • Step 204 Determine target map information corresponding to the obstacle to be predicted according to the motion trajectory.
  • the computer device searches for the lane centerline corresponding to the motion track, and the number of lane centerlines may be multiple.
  • the lane center line is sampled, and the lane center line is represented by a plurality of points obtained by sampling.
  • the multiple points obtained by sampling can be called location points. Therefore, the target map information corresponding to the obstacle to be predicted is obtained according to the center line of the lane.
  • the target map information may include a lane centerline corresponding to each motion track, and each motion track may correspond to multiple lane centerlines.
  • the lane centerline corresponding to each motion track can be referred to as a track map information, and a lane centerline can be referred to as a track lane information. Therefore, a track map information can contain multiple track lane information.
  • Each lane centerline includes a plurality of position points, and each position point includes coordinate values in the x-direction and the y-direction.
  • Step 206 converting the motion trajectory into a corresponding first trajectory matrix, and converting the target map information into a corresponding first map matrix.
  • the motion trajectory can be converted into a first trajectory matrix
  • the target map information can be converted into a corresponding first map matrix.
  • the first trajectory matrix is in the format of N_1 ⁇ T1 ⁇ 2, where N_1 represents the number of trajectories in the motion trajectory, T1 represents the number of trajectory points in each trajectory, and 2 represents the x and y coordinate directions.
  • the second map matrix is an N_2 ⁇ T2 ⁇ 2 matrix format, where N_2 represents the number of track lane information in the target map information, T2 represents the number of location points in each track lane information, and 2 represents the x and y coordinate directions.
  • Step 208 Input the first trajectory matrix and the first map matrix into the trained trajectory prediction model, perform embedding processing on the first trajectory matrix and the first map matrix to obtain the target matrix, and perform the target matrix based on the multi-head attention mechanism. Feature extraction, output features are obtained, and regression processing is performed on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • the target matrix refers to a matrix that identifies the positional relationship between the obstacle to be predicted and the corresponding target map information.
  • a trained trajectory prediction model is pre-stored in the computer device, and the trained trajectory prediction model is a model based on a multi-head attention network.
  • the multi-head attention network refers to the transformer network.
  • the trained trajectory prediction model is trained with a large amount of sample data.
  • Trained trajectory prediction models can include embedding networks, multi-head attention networks, and regression networks.
  • the embedding network may be composed of any existing one-dimensional convolutional network, and the embedding network is used to perform embedding processing on the first trajectory matrix and the first map matrix, and the embedding processing may include processing the first trajectory matrix and the first map matrix.
  • Position embedding refers to identifying the positional relationship between the obstacle to be predicted and the corresponding target map information.
  • the target matrix is used as the input of the multi-head attention network, and the feature extraction is performed on the target matrix through the multi-head attention network based on the multi-attention mechanism, and the output matrix is obtained.
  • the output matrix is a matrix obtained by concatenating matrices extracted by multiple attention heads.
  • the multi-attention mechanism refers to the feature extraction mechanism of the multi-head self-attention layer in the multi-head attention network, which can focus on the relationship between the obstacles to be predicted in the target matrix and the target map information from different positions. It can obtain richer and more comprehensive feature information and fully extract the deeper correlation between map information and obstacle information.
  • the regression network can be any of the existing one-dimensional convolutional neural networks.
  • the output matrix is input into the regression network, and the prediction operation is performed on the output matrix through the regression network to obtain the predicted trajectory of the obstacle to be predicted.
  • the predicted trajectory may be the motion trajectory of the obstacle to be predicted in the future, such as the motion trajectory in the next 3s.
  • the motion trajectory of the obstacle to be predicted is obtained, the target map information corresponding to the obstacle to be predicted is determined according to the motion trajectory, the motion trajectory is converted into a corresponding first trajectory matrix, and the target map information is converted into a corresponding first trajectory matrix.
  • the first map matrix of so that the motion trajectory and target map information meet the input requirements of the trajectory prediction model.
  • the first trajectory matrix and the first map matrix are input into the trained trajectory prediction model, the first trajectory matrix and the first map matrix are embedded to obtain the target matrix, and the feature extraction is performed on the target matrix based on the multi-head attention mechanism,
  • the output feature is obtained, and the output feature is subjected to regression processing to obtain the predicted trajectory of the obstacle to be detected.
  • the multi-head attention mechanism in the trajectory prediction model can pay attention to the relationship between the obstacles to be predicted in the target matrix and the target map information from different positions, more abundant and comprehensive feature information can be obtained, and the map information and obstacles can be fully extracted. Deeper correlation between object information improves the accuracy of trajectory prediction.
  • the motion trajectory and the target map information can be divided into information in the x direction and the y direction, so that the x direction It performs independent operation with the information in the y direction, which improves the efficiency of trajectory prediction.
  • acquiring the motion trajectory of the obstacle to be predicted includes: acquiring drive test data, performing perceptual processing on the drive test data, and obtaining trajectory information of the obstacle to be predicted in the drive test data; The trajectory information of the obstacle is sampled to obtain the motion trajectory corresponding to the obstacle to be predicted.
  • the road test data refers to the environmental information around the autonomous vehicle collected by the sensors during the autonomous driving process.
  • the computer equipment acquires the drive test data collected by the sensor, and performs perceptual processing on the drive test data.
  • Perceptual processing refers to target detection and target tracking on the drive test data.
  • the drive test data can be point cloud data or surrounding environment images.
  • the point cloud data can be detected by any target detection model, such as PointNet, PointPillar, PolarNet, Semantic Segment Models (semantic segmentation model), etc.
  • the coordinates of the center point represent the position information of the obstacle to be predicted.
  • target detection models can be used, such as SSD (Single Shot MultiBox Detector direct multi-target detection) model, RefineDet (Single-Shot Refinement neural network for Object Detection, fine direct multi-target detection), Mobilenet -SSD (Mobilenet based Single Shot MultiBox Detector, direct multi-target detection based on efficient convolutional neural network for mobile vision applications) model, YOLO (You Only Look Once, unified real-time target detection) model, etc.
  • the surrounding environment image is used for target detection, and the two-dimensional bounding box corresponding to the obstacle to be predicted is determined, including the coordinates, size, and orientation of the center point of the obstacle to be predicted.
  • the coordinates of the center point represent the position information of the obstacle to be predicted.
  • any one of the traditional trackers such as Kalman filter (KF), Unscented Kalman Filter (UKF) and other traditional trackers can be used to predict subsequent frames.
  • the speed information of the obstacle to be detected in By performing target detection and target tracking on each frame of drive test data, the trajectory information of the obstacle to be predicted in each frame is obtained. Since the trajectory information of the obstacle to be detected may be stationary or moving at a uniform speed, in order to improve the accuracy of the trajectory prediction, the trajectory information can be sampled, and only non-stationary or non-uniformly changing trajectories are sampled.
  • the computer equipment can sample and process the trajectory information of the obstacle to be predicted according to the preset sampling conditions, so as to obtain the motion trajectory of the obstacle to be predicted.
  • the preset sampling conditions refer to trajectories in the junction area, trajectories with changes in curvature and speed, and trajectories with lane changes and cut ins.
  • the motion trajectory includes a plurality of trajectory points, and each trajectory point includes coordinate values in the x-direction and the y-direction.
  • the track information of the obstacle to be predicted in the drive test data is obtained by perceptual processing of the drive test data, and sampling processing is performed on the track information of the obstacle to be predicted according to the preset sampling conditions, so as to obtain the corresponding information of the obstacle to be predicted. movement trajectory.
  • sampling processing is performed on the track information of the obstacle to be predicted according to the preset sampling conditions, so as to obtain the corresponding information of the obstacle to be predicted. movement trajectory.
  • determining the target map information corresponding to the obstacle to be predicted according to the motion trajectory includes: determining the corresponding lane center line according to the motion trajectory; sampling the lane center line to obtain the target map information corresponding to the obstacle to be predicted .
  • the position of the initial trajectory point of the motion trajectory For the motion trajectory of the obstacle to be predicted, determine the position of the initial trajectory point of the motion trajectory, and take the position as the center of the circle to determine a circular area with a radius of r, for example, r is 3m.
  • the computer device determines, based on the high-precision map, the lane centerlines that intersect with the circular area. There may be multiple lane centerlines that intersect, and the target map information corresponding to the motion trajectory is obtained according to the multiple lane centerlines that intersect. If the position of the initial trajectory point of the obstacle to be predicted is relatively close to the lane boundary (lane change may occur), then the lane centerline of the obstacle to be predicted includes the lane centerline where the initial trajectory point is located and the lane center to be changed. Wire.
  • the centerline of each lane corresponding to the motion trajectory can be uniformly sampled into N Points, that is, the sampled points represent each lane centerline, and each lane centerline includes N position points.
  • the number of sampling points can be set according to the duration of motion estimation and the duration of the trajectory to be predicted.
  • the corresponding lane centerline is determined according to the motion trajectory, and the lane centerline is sampled to obtain the target map information corresponding to the obstacle to be predicted, and the target map information related to the motion trajectory can be accurately obtained, which is conducive to improving the The accuracy of trajectory prediction.
  • the trained trajectory prediction model includes a multi-head attention network
  • the multi-head attention network includes a one-dimensional convolution layer
  • the one-dimensional convolution layer is used to perform feature extraction in the abscissa direction of the target matrix respectively and Feature extraction in the ordinate direction.
  • the trained trajectory prediction model includes sequentially connected embedding network, multi-head attention network and regression network.
  • the multi-head attention network is a transformer network, and " ⁇ N" indicates that the transformer network includes multiple multi-head attention layers and feed-forward neural network layers. There is an Add&Norm layer after the multi-head attention layer and the feed-forward neural network layer.
  • the multi-head attention layer extracts the feature of the target matrix through the multi-head attention mechanism, and the multi-head attention mechanism can pay attention to the trajectory points in different positions in the target matrix.
  • Inputting the computational paths of multiple trajectory points to the feedforward neural network layer makes the matrix-vector interactions in the multi-head attention network more interactive and can learn more complex relationships. Since the path has no dependencies in the feedforward unit, the output features can be obtained by executing the calculation path of multiple trajectory points in parallel through the feedforward neural network layer.
  • Add is a residual network, and the residual structure can eliminate the problem of information loss caused by deepening the number of layers.
  • Norm refers to Layer Normalization (layer normalization). Therefore, the Add&Norm unit is used to add and normalize the input and output of the multi-head attention layer or feedforward neural network layer.
  • Layer Normalization is used to convert the input into data with a mean of 0 and a variance of 1 to avoid the input falling into the saturation region of the subsequent activation function.
  • the traditional transformer network includes a Linear layer
  • the transformer network in this embodiment is an improved transformer network.
  • the specific method is to replace the Linear in the traditional transformer network with a one-dimensional convolution layer. Therefore, the feature extraction in the abscissa direction and the feature extraction in the ordinate direction can be performed on the target matrix through the one-dimensional convolution layer.
  • the data in the y direction is independently operated, which effectively improves the efficiency of trajectory prediction, and the accuracy of trajectory prediction is also improved.
  • the first trajectory matrix and the first map matrix are embedded, and the step of obtaining the target matrix includes:
  • Step 402 Perform feature extraction on the first trajectory matrix and the first map matrix respectively through the embedding network in the trained trajectory prediction model, obtain the channel number of the last convolutional layer of the embedding network, and obtain the first trajectory according to the channel data.
  • Step 404 Combine the first feature matrix and the second feature matrix to obtain a combined matrix.
  • Step 406 adding feature parameters to the combined matrix, and performing position embedding processing on the combined matrix after adding the feature parameters to obtain a target matrix.
  • the trained trajectory prediction model includes an embedding network, a multi-head attention network and a regression network, and the embedding network can be a one-dimensional convolutional network.
  • the embedding network is to convert the first trajectory matrix and the first map matrix into the matrix format required by the multi-head attention network, which can be used to capture the distance between the trajectory points in the first trajectory matrix and the position points in the first map matrix in a high-dimensional space Relationship.
  • the feature extraction is performed on the first trajectory matrix and the first map matrix respectively through the embedding network, and the first feature matrix corresponding to the first trajectory matrix and the first feature matrix corresponding to the first map matrix are generated according to the number of channels of the last convolutional layer of the embedding network. Two feature matrices.
  • the number of channels in the last convolutional layer can be represented by dim1.
  • the first feature matrix can be represented as N_1 ⁇ dim1 ⁇ 2, where N_1 represents the number of tracks in the first feature matrix, and 2 represents the x and y coordinate directions.
  • the second feature matrix may be represented as N_2 ⁇ dim1 ⁇ 2, where N_2 represents the number of track lane information in the second feature matrix, and 2 represents the x and y coordinate directions.
  • the first feature matrix and the second feature matrix are combined in the second dimension through the embedding network to obtain a combined matrix.
  • the combined matrix is a four-dimensional matrix.
  • the combined matrix can be expressed as N_1 ⁇ dim2 ⁇ dim_1 ⁇ 2, where dim2 represents After the first feature matrix and the second feature matrix are combined in the second dimension, the total number of features in the second dimension. dim2 can be preset, so that the computer device can combine the first feature matrix and the second feature matrix according to the preset value. In the merging process, each obstacle to be predicted is traversed.
  • the track lane information in the second feature matrix Randomly select dim2-1 track lane information from the data and merge the track of the obstacle to be predicted in the first feature matrix in the second dimension; if the number of track lane information corresponding to the track of the obstacle to be predicted in the second feature matrix +1 is less than dim2, you need to stack 0 matrices in the second dimension, so that the total number of features of the combined second dimension is dim2.
  • the combined matrix after adding the feature parameters can be expressed as N_1 ⁇ (1+dim2) ⁇ dim_1 ⁇ 2, where 1 represents the added feature parameter, which can be is an arbitrary numerical value.
  • Feature parameters are used to collect information on the map and obstacles to be predicted at scale for subsequent trajectory prediction.
  • the combined matrix after adding the feature parameters can be processed by position embedding to obtain the target matrix.
  • the positional relationship between the obstacles to be predicted and the map information in the matrix can be identified by position embedding, which is used to make up for the lack of positional information.
  • the target matrix can be directly input into the multi-head attention network for feature extraction.
  • feature extraction is performed on the first trajectory matrix and the first map matrix respectively through the embedding network, and the first feature matrix and the first feature matrix corresponding to the first trajectory matrix are obtained according to the number of channels of the last convolutional layer of the embedding network.
  • the second feature matrix corresponding to a map matrix can obtain the matrix format required by the multi-head attention network, and can be used to capture the relationship between the trajectory points in the first trajectory matrix and the position points in the first map matrix in a high-dimensional space.
  • the first feature matrix and the second feature matrix are combined to obtain a combined matrix, and feature parameters are added to the combined matrix, which can quickly collect information on the map and obstacles to be predicted for subsequent trajectory prediction.
  • Combining the matrix for position embedding processing can make up for the lack of position information between the obstacles to be predicted and the map information in the multi-head attention network, and can further improve the accuracy of trajectory prediction.
  • the method before acquiring the motion trajectory of the obstacle to be predicted, the method further includes: acquiring a training sample, where the training sample includes trajectory information of the target obstacle and sample map information corresponding to the target obstacle; converting the trajectory information into Be the corresponding second trajectory matrix, and convert the sample map information into the corresponding second map matrix; input the second trajectory matrix and the second map matrix into the trajectory prediction model to be trained, and output the future trajectory of the target obstacle;
  • the model loss of the trajectory prediction model to be trained is calculated according to the trajectory information and the future trajectory, and the model parameters of the trajectory prediction model to be trained are updated according to the model loss until the preset conditions are met, and the trained trajectory prediction model is obtained.
  • the training sample refers to the sample data used to train the trajectory prediction model, and the training sample includes the trajectory information of the target obstacle and the sample map information corresponding to the target obstacle.
  • the target obstacle refers to dynamic obstacles, such as vehicles, pedestrians, etc.
  • the computer device obtains the historical drive test data collected by the sensor, performs perception processing on the historical drive test data, and obtains the trajectory information of the dynamic obstacles in the historical drive test data.
  • Perceptual processing refers to target detection and target tracking, which is the same as the perceptual processing method in the application process of the above trajectory prediction model, and will not be repeated here.
  • the computer device performs sampling processing on the trajectory information of the dynamic obstacle according to the preset sampling conditions, and obtains a trajectory sample set corresponding to the dynamic obstacle.
  • the preset sampling conditions can be a trajectory in a junction area, a trajectory with a change in curvature and speed, a trajectory with a lane change and a cut in.
  • the trajectory sample set includes historical trajectories of multiple dynamic obstacles, and each historical trajectory includes multiple trajectory points.
  • each historical trajectory may include 50 trajectory points.
  • Each track point includes x-direction and y-direction coordinate values.
  • the track lane information corresponding to the dynamic obstacle is determined according to each historical track in the track sample set, and the map sample set corresponding to the dynamic obstacle is obtained.
  • the sampling method of the trajectory lane information is the same as the sampling method in the application process of the above trajectory prediction model, and will not be repeated here.
  • the trajectory information and sample map information corresponding to the target obstacle are respectively selected from the trajectory sample set and the map sample set to generate training samples.
  • the trajectory sample set and the map sample set can be divided into training samples, test sets and validation sets according to a preset ratio.
  • the preset ratio can be 3:1:1.
  • the purpose of dividing the trajectory sample set and the map sample set into three sets is to select the model with the highest accuracy and the best generalization ability.
  • the training process of the trajectory prediction model is the same as the trajectory prediction method in the application process, that is, the trajectory information in the training sample is converted into the corresponding second trajectory matrix, and the sample map information in the training sample is converted into the corresponding second map matrix, input the second trajectory matrix and the second map matrix into the trajectory prediction model to be trained, and output the future trajectory of the target obstacle.
  • the model loss of the trajectory prediction model to be trained is calculated according to the trajectory information and the future trajectory, and the model parameters are adjusted according to the model loss to obtain the trained trajectory prediction model.
  • the model loss can be existing loss functions such as MSE mean square error loss, cross entropy loss, etc.
  • the model parameters are adjusted through the output backpropagation of the loss function.
  • model training process is an iterative training process, it needs to go through multiple epoch, 1 epoch means that all training samples are used for training once, and each epoch will output a model parameter.
  • the model parameters with the highest accuracy can be determined through the validation set, that is, to determine which epoch outputs the model parameters to obtain a more accurate future trajectory.
  • the specific judgment method can be to determine whether the network loss value reaches the loss threshold, or whether the number of iterations reaches the iteration number.
  • the number of times threshold if the network loss value reaches the loss threshold, or the number of iterations reaches the threshold of the number of iterations, the model parameters output by the corresponding epoch can be used as the final model parameters, and the model is the trained trajectory prediction model.
  • a training sample is obtained, the training sample includes trajectory information of the target obstacle and sample map information corresponding to the target obstacle, the trajectory information is converted into a corresponding second trajectory matrix, and the sample map information is converted into a corresponding
  • the second map matrix, the second trajectory matrix and the second map matrix are input into the trajectory prediction model to be trained, the model loss of the trajectory prediction model to be trained is calculated, and the model parameters of the trajectory prediction model to be trained are updated according to the model loss, Get the trained trajectory prediction model. Since the multi-head attention mechanism in the trajectory prediction model can pay attention to the relationship between the target obstacle and the sample map information in the target matrix from different positions, it can obtain more abundant and comprehensive feature information, and fully extract map information and obstacles. The deeper correlation between information improves the accuracy of trajectory prediction.
  • a trajectory prediction apparatus including: a trajectory acquisition module 502, a map acquisition module 504, a matrix conversion module 506, and a trajectory prediction module 508, wherein:
  • the trajectory acquisition module 502 is used to acquire the motion trajectory of the obstacle to be predicted.
  • the map acquisition module 504 is configured to determine target map information corresponding to the obstacle to be predicted according to the motion trajectory.
  • the matrix conversion module 506 is configured to convert the motion trajectory into a corresponding first trajectory matrix, and convert the target map information into a corresponding first map matrix.
  • the trajectory prediction module 508 is configured to input the first trajectory matrix and the first map matrix into the trained trajectory prediction model, perform embedding processing on the first trajectory matrix and the first map matrix, and obtain the target matrix, based on the multi-head attention mechanism Perform feature extraction on the target matrix to obtain output features, and perform regression processing on the output features to obtain the predicted trajectory of the obstacle to be detected.
  • the trained trajectory prediction model includes a multi-head attention network
  • the multi-head attention network includes a one-dimensional convolutional layer
  • the trajectory prediction module 508 is further configured to perform a horizontal cross-section on the target matrix according to the one-dimensional convolutional layer. Feature extraction in the coordinate direction and feature extraction in the ordinate direction.
  • the trajectory prediction module 508 is further configured to perform feature extraction on the first trajectory matrix and the first map matrix respectively through the embedding network in the trained trajectory prediction model to obtain the last convolutional layer of the embedding network According to the number of channels, the first feature matrix corresponding to the first trajectory matrix and the second feature matrix corresponding to the first map matrix are obtained according to the number of channels; the first feature matrix and the second feature matrix are merged to obtain a combined matrix; in the combined matrix Add feature parameters to , and perform position embedding processing on the combined matrix after adding feature parameters to obtain the target matrix.
  • the trajectory acquisition module 508 is further configured to acquire drive test data, perform perceptual processing on the drive test data, and obtain trajectory information of obstacles to be predicted in the drive test data; The trajectory information is sampled to obtain the motion trajectory corresponding to the obstacle to be predicted.
  • the map acquisition module 504 is further configured to determine the corresponding lane centerline according to the motion trajectory; perform sampling processing on the lane centerline to obtain target map information corresponding to the obstacle to be predicted.
  • the above-mentioned device further includes:
  • the sample acquisition module is used to acquire training samples, and the training samples include the trajectory information of the target obstacle and the sample map information corresponding to the target obstacle.
  • the sample conversion module is configured to convert the trajectory information into a corresponding second trajectory matrix, and convert the sample map information into a corresponding second map matrix.
  • the trajectory calculation module is used to input the second trajectory matrix and the second map matrix into the trajectory prediction model to be trained, and output the future trajectory of the target obstacle.
  • the parameter updating module is used to calculate the model loss of the trajectory prediction model to be trained according to the trajectory information and future trajectories, update the model parameters of the trajectory prediction model to be trained according to the model loss, and obtain the trained trajectory prediction model.
  • Each module in the above-mentioned trajectory prediction apparatus can be implemented in whole or in part by software, hardware and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device is provided, the internal structure of which can be shown in FIG. 6 .
  • the computer device includes a processor, memory, a communication interface, and a database connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer device is used to store data for a trajectory prediction method.
  • the communication interface of the computer device is used to connect and communicate with an external terminal.
  • the computer readable instructions when executed by a processor, implement a trajectory prediction method.
  • FIG. 6 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device comprising a memory and one or more processors, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, makes the one or more processors execute the above methods to implement steps in the example.
  • One or more computer storage media storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the steps in each of the foregoing method embodiments.
  • the computer storage medium is a readable storage medium, and the readable storage medium may be non-volatile or volatile.
  • a vehicle is provided, the vehicle may specifically include an autonomous driving vehicle, and the vehicle includes the above computer device, which can execute the steps in the above embodiment of the trajectory prediction method.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Procédé de prédiction de trajectoire comprenant les étapes consistant à : acquérir une trajectoire de mouvement d'un obstacle à soumettre à une prédiction (202) ; en fonction de la trajectoire de mouvement, déterminer des informations de carte cibles correspondant audit obstacle (204) ; convertir la trajectoire de mouvement en une première matrice de trajectoire correspondante, et convertir les informations de carte cibles en une première matrice de carte correspondante (206) ; et entrer la première matrice de trajectoire et la première matrice de carte dans un modèle de prédiction de trajectoire entraîné, effectuer un traitement d'incorporation sur la première matrice de trajectoire et la première matrice de carte, de manière à obtenir une matrice cible, effectuer une extraction d'élément sur la matrice cible sur la base d'un mécanisme d'attention multi-tête, de manière à obtenir un élément de sortie, et effectuer un traitement de régression sur l'élément de sortie, de manière à obtenir une trajectoire prédite dudit obstacle (208).
PCT/CN2021/088937 2021-04-22 2021-04-22 Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage WO2022222095A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/088937 WO2022222095A1 (fr) 2021-04-22 2021-04-22 Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage
CN202180050157.3A CN115917559A (zh) 2021-04-22 2021-04-22 轨迹预测方法、装置、计算机设备和存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/088937 WO2022222095A1 (fr) 2021-04-22 2021-04-22 Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage

Publications (1)

Publication Number Publication Date
WO2022222095A1 true WO2022222095A1 (fr) 2022-10-27

Family

ID=83723549

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/088937 WO2022222095A1 (fr) 2021-04-22 2021-04-22 Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage

Country Status (2)

Country Link
CN (1) CN115917559A (fr)
WO (1) WO2022222095A1 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115047894A (zh) * 2022-04-14 2022-09-13 中国民用航空总局第二研究所 一种无人机轨迹测算方法、电子设备及存储介质
CN116071925A (zh) * 2023-02-13 2023-05-05 北京爱芯科技有限公司 轨迹预测方法、装置及电子处理装置
CN116184352A (zh) * 2023-04-26 2023-05-30 武汉能钠智能装备技术股份有限公司四川省成都市分公司 基于轨迹估计的射频目标侦测系统
CN116341721A (zh) * 2023-03-02 2023-06-27 石家庄铁道大学 一种基于车辆轨迹的运行目的地预测方法及系统
CN116824407A (zh) * 2023-06-21 2023-09-29 深圳市华赛睿飞智能科技有限公司 一种基于巡逻机器人的目标检测方法、装置及设备
CN117058472A (zh) * 2023-10-12 2023-11-14 华侨大学 基于自注意力机制的3d目标检测方法、装置、设备
CN117473337A (zh) * 2023-12-25 2024-01-30 中国科学院自动化研究所 轨迹关联方法及装置、设备及存储介质
CN117493424A (zh) * 2024-01-03 2024-02-02 湖南工程学院 一种不依赖地图信息的车辆轨迹预测方法
CN117874529A (zh) * 2024-03-12 2024-04-12 浪潮电子信息产业股份有限公司 运动轨迹预测方法、模型训练方法、装置、设备及介质
CN117874529B (zh) * 2024-03-12 2024-05-28 浪潮电子信息产业股份有限公司 运动轨迹预测方法、模型训练方法、装置、设备及介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116558540B (zh) * 2023-07-11 2023-10-03 新石器慧通（北京）科技有限公司 模型训练方法和装置、轨迹生成方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111238523A (zh) * 2020-04-23 2020-06-05 北京三快在线科技有限公司 一种运动轨迹的预测方法及装置
US20200283016A1 (en) * 2019-03-06 2020-09-10 Robert Bosch Gmbh Movement prediction of pedestrians useful for autonomous driving
CN112015847A (zh) * 2020-10-19 2020-12-01 北京三快在线科技有限公司 一种障碍物的轨迹预测方法、装置、存储介质及电子设备
CN112037506A (zh) * 2020-07-29 2020-12-04 长安大学 车辆轨迹预测模型构建方法、车辆轨迹预测方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200283016A1 (en) * 2019-03-06 2020-09-10 Robert Bosch Gmbh Movement prediction of pedestrians useful for autonomous driving
CN111238523A (zh) * 2020-04-23 2020-06-05 北京三快在线科技有限公司 一种运动轨迹的预测方法及装置
CN112037506A (zh) * 2020-07-29 2020-12-04 长安大学 车辆轨迹预测模型构建方法、车辆轨迹预测方法及系统
CN112015847A (zh) * 2020-10-19 2020-12-01 北京三快在线科技有限公司 一种障碍物的轨迹预测方法、装置、存储介质及电子设备

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115047894B (zh) * 2022-04-14 2023-09-15 中国民用航空总局第二研究所 一种无人机轨迹测算方法、电子设备及存储介质
CN115047894A (zh) * 2022-04-14 2022-09-13 中国民用航空总局第二研究所 一种无人机轨迹测算方法、电子设备及存储介质
CN116071925B (zh) * 2023-02-13 2024-04-12 北京爱芯科技有限公司 轨迹预测方法、装置及电子处理装置
CN116071925A (zh) * 2023-02-13 2023-05-05 北京爱芯科技有限公司 轨迹预测方法、装置及电子处理装置
CN116341721A (zh) * 2023-03-02 2023-06-27 石家庄铁道大学 一种基于车辆轨迹的运行目的地预测方法及系统
CN116341721B (zh) * 2023-03-02 2023-10-31 石家庄铁道大学 一种基于车辆轨迹的运行目的地预测方法及系统
CN116184352A (zh) * 2023-04-26 2023-05-30 武汉能钠智能装备技术股份有限公司四川省成都市分公司 基于轨迹估计的射频目标侦测系统
CN116184352B (zh) * 2023-04-26 2023-08-22 武汉能钠智能装备技术股份有限公司四川省成都市分公司 基于轨迹估计的射频目标侦测系统
CN116824407A (zh) * 2023-06-21 2023-09-29 深圳市华赛睿飞智能科技有限公司 一种基于巡逻机器人的目标检测方法、装置及设备
CN117058472A (zh) * 2023-10-12 2023-11-14 华侨大学 基于自注意力机制的3d目标检测方法、装置、设备
CN117058472B (zh) * 2023-10-12 2024-02-20 华侨大学 基于自注意力机制的3d目标检测方法、装置、设备
CN117473337B (zh) * 2023-12-25 2024-03-22 中国科学院自动化研究所 轨迹关联方法及装置、设备及存储介质
CN117473337A (zh) * 2023-12-25 2024-01-30 中国科学院自动化研究所 轨迹关联方法及装置、设备及存储介质
CN117493424A (zh) * 2024-01-03 2024-02-02 湖南工程学院 一种不依赖地图信息的车辆轨迹预测方法
CN117493424B (zh) * 2024-01-03 2024-03-22 湖南工程学院 一种不依赖地图信息的车辆轨迹预测方法
CN117874529A (zh) * 2024-03-12 2024-04-12 浪潮电子信息产业股份有限公司 运动轨迹预测方法、模型训练方法、装置、设备及介质
CN117874529B (zh) * 2024-03-12 2024-05-28 浪潮电子信息产业股份有限公司 运动轨迹预测方法、模型训练方法、装置、设备及介质

Also Published As

Publication number Publication date
CN115917559A (zh) 2023-04-04

Similar Documents

Publication Publication Date Title
WO2022222095A1 (fr) Procédé et appareil de prédiction de trajectoire, dispositif informatique et support de stockage
CN108171112B (zh) 基于卷积神经网络的车辆识别与跟踪方法
CN110059558B (zh) 一种基于改进ssd网络的果园障碍物实时检测方法
CN112149550B (zh) 一种基于多传感器融合的自动驾驶车辆3d目标检测方法
WO2021134296A1 (fr) Procédé et appareil de détection d'obstacles, et dispositif informatique et support de stockage
US20230316742A1 (en) Image processing method, apparatus and device, and computer-readable storage medium
US20230222844A1 (en) Parking lot management and control method based on object activity prediction, and electronic device
Loktev et al. Diagnostics of external defects of railway infrastructure by analysis of its images
CN113592905B (zh) 基于单目摄像头的车辆行驶轨迹预测方法
CN112634329B (zh) 一种基于时空与或图的场景目标活动预测方法及装置
WO2021223116A1 (fr) Procédé et appareil de génération de carte de perception, dispositif informatique, et support de stockage
CN111292366A (zh) 一种基于深度学习和边缘计算的视觉行车测距算法
CN115204044A (zh) 轨迹预测模型的生成以及轨迹信息处理方法、设备及介质
CN112967388A (zh) 三维时序图像神经网络模型的训练方法和装置
Khurana et al. Point cloud forecasting as a proxy for 4d occupancy forecasting
CN114648551B (zh) 轨迹预测方法及装置
CN111753639A (zh) 感知地图生成方法、装置、计算机设备和存储介质
CN111402632B (zh) 一种交叉口行人运动轨迹的风险预测方法
CN115512175A (zh) 模型训练、点云数据处理方法、装置、设备及存储介质
Kanchana et al. Computer vision for autonomous driving
CN112613418A (zh) 基于目标活动预测的停车场管控方法、装置及电子设备
BOURJA et al. Real time vehicle detection, tracking, and inter-vehicle distance estimation based on stereovision and deep learning using YOLOv3
JP2022035033A (ja) 情報処理システム、情報処理方法、プログラムおよび車両制御システム
CN113361299A (zh) 一种异常停车的检测方法、装置、存储介质及电子设备
Abdelhalim et al. Vt-lane: An exploratory study of an ad-hoc framework for real-time intersection turn count and trajectory reconstruction using nema phases-based virtual traffic lanes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21937334

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE