WO2022226837A1 - 基于时间与空间学习的轨迹预测方法、装置和计算机设备 - Google Patents

基于时间与空间学习的轨迹预测方法、装置和计算机设备 Download PDF

Info

Publication number
WO2022226837A1
WO2022226837A1 PCT/CN2021/090552 CN2021090552W WO2022226837A1 WO 2022226837 A1 WO2022226837 A1 WO 2022226837A1 CN 2021090552 W CN2021090552 W CN 2021090552W WO 2022226837 A1 WO2022226837 A1 WO 2022226837A1
Authority
WO
WIPO (PCT)
Prior art keywords
matrix
map
predicted
feature
obstacle
Prior art date
Application number
PCT/CN2021/090552
Other languages
English (en)
French (fr)
Inventor
许家妙
何明
Original Assignee
深圳元戎启行科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳元戎启行科技有限公司 filed Critical 深圳元戎启行科技有限公司
Priority to CN202180050155.4A priority Critical patent/CN115943400B/zh
Priority to PCT/CN2021/090552 priority patent/WO2022226837A1/zh
Publication of WO2022226837A1 publication Critical patent/WO2022226837A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models

Definitions

  • the present application relates to a trajectory prediction method, device, computer equipment, storage medium and vehicle based on time and space learning.
  • trajectory prediction can be carried out through deep learning-based trajectory prediction methods, such as preprocessing the historical trajectory information and map data of obstacles into raster images or vectorized data, and then using deep networks to process raster images or vectorized data. .
  • the historical trajectory information of obstacles can be called time information, and the relationship between the historical trajectory information of obstacles and map data can be called spatial information. Since temporal information and spatial information are particularly important for the trajectory prediction of obstacles, the existing trajectory prediction methods based on deep learning cannot make full use of temporal information and spatial information at the same time, resulting in low trajectory prediction accuracy.
  • a method, apparatus, computer device, storage medium and vehicle for trajectory prediction based on time and space learning are provided.
  • a trajectory prediction method based on time and space learning including:
  • the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted;
  • the target matrix is input into the time information model, and the first characteristic matrix corresponding to the position matrix and the second characteristic matrix corresponding to the map matrix are obtained;
  • the spatial feature matrix is input into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • a trajectory prediction device based on time and space learning comprising:
  • the data acquisition module is used to acquire the preset frame position data of the obstacle to be predicted, and the map data;
  • a matrix generation module configured to generate a target matrix according to the preset frame position data and the map data, where the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted ;
  • a time information extraction module for inputting the target matrix into a time information model to obtain a first feature matrix corresponding to the position matrix and a second feature matrix corresponding to the map matrix;
  • a spatial information integration module configured to perform spatial information integration of the first feature matrix and the second feature matrix to obtain a spatial feature matrix
  • the trajectory prediction module is used for inputting the spatial feature matrix into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • a computer device comprising a memory and one or more processors, the memory having computer-readable instructions stored therein, the computer-readable instructions, when executed by the processor, cause the one or more processors to execute The following steps:
  • the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted;
  • the target matrix is input into the time information model, and the first characteristic matrix corresponding to the position matrix and the second characteristic matrix corresponding to the map matrix are obtained;
  • the spatial feature matrix is input into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • One or more computer storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:
  • the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted;
  • the target matrix is input into the time information model, and the first characteristic matrix corresponding to the position matrix and the second characteristic matrix corresponding to the map matrix are obtained;
  • the spatial feature matrix is input into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • a vehicle comprising the steps of executing the above-mentioned trajectory prediction method based on time and space learning.
  • FIG. 1 is an application environment diagram of a trajectory prediction method based on time and space learning in one or more embodiments.
  • FIG. 2 is a schematic flowchart of a trajectory prediction method based on temporal and spatial learning in one or more embodiments.
  • FIG. 3 is a schematic diagram of a lane line map obtained by searching for associated lane lines in one or more embodiments.
  • FIG. 4 is a schematic flowchart of steps of inputting a target matrix into a time information model to obtain a first feature matrix corresponding to the position matrix and a second feature matrix corresponding to the map matrix in one or more embodiments.
  • FIG. 5 is a schematic flowchart of a step of integrating spatial information of a first feature matrix and a second feature matrix to obtain a spatial feature matrix in one or more embodiments.
  • FIG. 6 is a block diagram of a trajectory prediction apparatus based on temporal and spatial learning in one or more embodiments.
  • FIG. 7 is a block diagram of a computer device in one or more embodiments.
  • the trajectory prediction method based on time and space learning provided in this application can be applied to the application environment shown in FIG. 1 .
  • the onboard sensor 102 communicates with the onboard computer device 104 over a network.
  • the number of in-vehicle sensors can be one or more.
  • the in-vehicle computer equipment may be simply referred to as computer equipment.
  • the vehicle-mounted sensor 102 sends the collected point cloud data to the computer device 104, and the computer device 104 performs target detection on the point cloud data, obtains the preset frame position data of the obstacle to be predicted, and obtains the pre-stored map data.
  • the in-vehicle sensor 102 can be, but is not limited to, a lidar, a laser scanner, or a camera.
  • the in-vehicle computer device 104 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, and can also be implemented by an independent server or a server cluster composed of multiple servers.
  • a trajectory prediction method based on time and space learning is provided, and the method is applied to the computer device in FIG. 1 as an example to illustrate, including the following steps:
  • Step 202 Acquire preset frame position data and map data of the obstacle to be predicted.
  • the obstacles to be predicted refer to the dynamic obstacles around the unmanned vehicle during the driving process.
  • the obstacles to be predicted may include pedestrians, vehicles, and the like.
  • the preset frame position data refers to the position of the obstacle to be predicted in multiple consecutive frames in history, including the position of the current frame.
  • Map data refers to high-precision maps pre-stored in computer equipment.
  • the precision map contains rich and detailed road traffic information elements.
  • High-precision maps not only have high-precision coordinates, but also include accurate road shapes, and also include data on the slope, curvature, heading, elevation, roll, etc. of each lane.
  • High-resolution maps not only depict roads, but also the lane lines that exist on each road.
  • the on-board sensors installed on the vehicle can send the collected point cloud data to the computer equipment.
  • Computer equipment can save point cloud data in units of frames, and record the data collection time and other information of each frame of point cloud data.
  • the vehicle sensor can be a lidar, a laser scanner, a camera, and the like.
  • the computer equipment can perform trajectory prediction in real time. For the current frame, the computer equipment can obtain the point cloud data of the preset frame including the point cloud data of the current frame, and perform target detection on the point cloud data of the preset frame to determine the The position of the obstacle in the world coordinate system is predicted, so as to obtain the preset frame position data of the obstacle to be predicted.
  • the preset number of frames may be preset, and similarly, the number of predicted frames may also be preset, and the number of predicted frames refers to the number of frames corresponding to the target trajectory obtained by the trajectory prediction.
  • the frequency of the lidar is 10Hz
  • the target trajectory within 3s needs to be predicted according to the trajectory data of the unmanned vehicle within 2s.
  • the position of the obstacle to be predicted in the world coordinate system can be represented by (x, y). Therefore, the preset frame position data includes the data in the abscissa direction (x direction) and the ordinate direction (y direction). )The data.
  • Step 204 Generate a target matrix according to the preset frame position data and the map data, where the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted.
  • the target matrix refers to a matrix obtained by integrating the preset frame data and the map data.
  • the computer device may first convert the preset frame position data into a corresponding position matrix, where the position matrix includes the position of the obstacle to be predicted in each frame.
  • the associated lane lines of the obstacles to be predicted are searched in the map data, and the associated lane lines refer to the lanes where the obstacles to be predicted may travel in the future.
  • the map matrix corresponding to the obstacle to be predicted is obtained according to the associated lane line.
  • the map matrix includes multiple lane lines corresponding to the obstacles to be predicted and the position of the lane line point corresponding to each associated lane line, and then the position matrix and the map matrix are combined into a matrix to obtain the target matrix.
  • Step 206 the target matrix is input into the time information model, and the first feature matrix corresponding to the position matrix and the second feature matrix corresponding to the map matrix are obtained.
  • the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted.
  • the first feature matrix refers to a matrix including time information hidden in the position matrix, that is, including time information embodied by the position data of the obstacle to be predicted.
  • the second feature matrix refers to a matrix including the time information hidden in the map matrix, that is, including the time information embodied by the associated lane lines of the obstacles to be predicted.
  • a time information model is pre-stored in the computer device, and the time information model is obtained by training a large amount of sample data.
  • the temporal information model may be a convolutional neural network model. Input the target matrix into the time information model, and process the frame number channel of the position matrix and the map matrix in the target matrix through the time information model, so as to learn the time information hidden in the frame number channel, that is, the time extracted to the position matrix and the map matrix.
  • the first feature matrix corresponding to the location matrix and the second feature matrix corresponding to the map matrix are respectively obtained according to the extracted time features.
  • Step 208 Integrate the spatial information of the first feature matrix and the second feature matrix to obtain a spatial feature matrix.
  • spatial information may be integrated between the first feature matrix corresponding to the position matrix and the second feature matrix corresponding to the map matrix.
  • the computer device can calculate the similarity between the first feature matrix and the second feature matrix, so as to obtain a new map feature according to the similarity and the second feature matrix corresponding to the map matrix, as the third feature matrix, Then, the third feature matrix and the first feature matrix are combined to realize the connection between the new map feature and the position feature of the obstacle to be predicted, so as to obtain a spatial feature matrix.
  • the spatial feature matrix is used to represent the relationship between the preset frame position data of the obstacle to be predicted and the lane line information.
  • Step 210 Input the spatial feature matrix into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • a trajectory prediction model is pre-stored in the computer device, and the trajectory prediction model and the above-mentioned time information model may be obtained by training with the same sample data.
  • the trajectory prediction model may be an Encode-Decode network model, specifically a convolutional neural network model.
  • the trajectory prediction model is used to predict the trajectory of the spatial feature matrix, and the target trajectory is output.
  • back-propagation algorithms such as SGD (Stochastic Gradient Descent, stochastic gradient descent), Adam (Adaptive Moment Estimation, automatic
  • SGD Spochastic Gradient Descent, stochastic gradient descent
  • Adam Adaptive Moment Estimation, automatic
  • the above model is trained by optimization methods such as adaptive moment estimation) algorithm, and the model parameters are obtained, and the time information model and the trajectory prediction model are stored with the corresponding model parameters, and the trained time information model and the trained trajectory prediction model are obtained.
  • the preset frame position data and map data of the obstacle to be predicted are acquired, and a target matrix is generated according to the preset frame position data and the map data, and the target matrix includes the preset frame position
  • the position matrix corresponding to the data and the map matrix corresponding to the obstacle to be predicted can obtain data that meets the input requirements of the model, and can also reduce the number of matrices, which facilitates the subsequent integration of time information and spatial information.
  • the target matrix is input into the time information model, the first feature matrix corresponding to the position matrix and the second feature matrix corresponding to the map matrix are obtained, and the first feature matrix and the second feature matrix are performed.
  • the spatial information is integrated to obtain a spatial feature matrix, and the spatial feature matrix is input into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • the temporal information and spatial information of the obstacle to be predicted in the preset frame can be fully utilized, and the accuracy of the trajectory prediction result is improved.
  • the position matrix is a matrix marked with the abscissa direction and the ordinate direction corresponding to the preset frame position data
  • the map matrix is marked with the abscissa direction and the ordinate direction of the associated lane line corresponding to the preset frame position data.
  • the matrix of the coordinate direction, the time information model and the trajectory prediction model are all one-dimensional convolutional neural network models.
  • the time information model and the trajectory prediction model can be the same type of convolutional neural network model or different types of convolutional neural networks. Model.
  • the computer device can mark the abscissa direction and the ordinate direction corresponding to the preset frame position data in the position matrix, and mark the abscissa direction and the ordinate direction of the associated lane line corresponding to the preset frame position data in the map matrix.
  • the abscissa data of the position of the obstacle to be predicted in each frame in the position matrix can be marked in the abscissa direction
  • the ordinate data can be marked in the ordinate direction, so as to distinguish the data in the abscissa direction and the ordinate direction.
  • the position matrix can be expressed as an M ⁇ 2-dimensional matrix [[x_ ⁇ -M+1 ⁇ ,y_ ⁇ -M+1 ⁇ ],[x_ ⁇ -M+2 ⁇ ,y_ ⁇ - M+2 ⁇ ],...,[x_ ⁇ 0 ⁇ ,y_ ⁇ 0 ⁇ ], where [x_ ⁇ -M+1 ⁇ ,y_ ⁇ -M+1 ⁇ ] is the historical M-1th frame to be predicted
  • [x_ ⁇ -M+1 ⁇ ,y_ ⁇ -M+1 ⁇ ] is the historical M-1th frame to be predicted
  • [x_ ⁇ -M+1 ⁇ ,y_ ⁇ -M+1 ⁇ ] is the historical M-1th frame to be predicted
  • the position of the obstacle in the world coordinate system [x_ ⁇ 0 ⁇ , y_ ⁇ 0 ⁇ ] represents the position of the obstacle to be predicted in the current frame in the world coordinate system
  • 2 represents the x coordinate direction and the y coordinate direction.
  • the map matrix includes multiple associated lane lines corresponding to the obstacles to be predicted, and the position of the lane line point corresponding to each associated lane line.
  • the abscissa data of the position of the lane line point can be marked in the abscissa direction, and the ordinate data Marking the ordinate direction is used to distinguish the data in the abscissa direction and the data in the ordinate direction.
  • the map matrix is a K ⁇ (N+M) ⁇ 2-dimensional matrix, where K represents the number of associated lane lines.
  • N represents the number of predicted frames
  • M represents the number of preset frames
  • (N+M) ⁇ 2 represents the position of the N+M points of a certain associated lane line in the world coordinate system.
  • Both the time information model and the trajectory prediction model can be a convolutional neural network model with a kernel size of 1 and a stride of 1, so that the data in the abscissa direction and the data in the ordinate direction are processed separately. , do not interfere with each other, can also reduce the amount of calculation, and the calculation speed is faster.
  • generating the target matrix according to the preset frame position data and the map data includes: converting the preset frame position data of the obstacle to be predicted into a position matrix; determining the obstacle to be predicted according to the preset frame position data and the map data The map matrix corresponding to the object; the position matrix and the map matrix are combined to obtain the target matrix.
  • the preset frame position data includes the position coordinates of the obstacle to be predicted in each frame.
  • the position matrix may be a matrix marked with the abscissa direction and the ordinate direction corresponding to the preset frame position data.
  • the map data is searched for the associated lane line of the obstacle to be predicted, and the associated lane line refers to the lane line where the obstacle to be predicted may travel after the initial frame position.
  • the associated lane lines can be sampled into multiple points, thereby obtaining a lane line point set, and converting the lane line point set into a map matrix.
  • the abscissa direction and the ordinate direction of the associated lane line corresponding to the preset frame position data are marked in the map matrix. Merge the position matrix with the map matrix to get the target matrix.
  • the preset frame position data of the obstacle to be predicted is converted into a position matrix
  • the map matrix corresponding to the obstacle to be predicted is determined according to the preset frame position data and the map data
  • the position matrix of the data in the ordinate direction and the map matrix are beneficial to the subsequent separate processing of the data in the abscissa direction and the data in the ordinate direction, which reduces the amount of calculation and makes the calculation speed faster.
  • Combining the position matrix and the map matrix to obtain the target matrix can reduce the number of matrices and facilitate the subsequent integration of time information and spatial information.
  • determining the map matrix corresponding to the obstacle to be predicted according to the preset frame position data and the map data includes: searching the map data for the associated lane line of the obstacle to be predicted according to the preset frame position data; sampling the associated lane line , get the lane line point set; convert the lane line point set into the map matrix corresponding to the obstacle to be predicted.
  • the KNN K-Nearest Neighbor, K nearest neighbor
  • the KNN K-Nearest Neighbor, K nearest neighbor
  • the length of the searched associated lane line can be V*T*(N+M), where V represents the average speed of the obstacle to be predicted in the preset frame, which can be calculated according to the preset frame position data, and T represents the adjacent frame
  • the time interval between position data such as 100 milliseconds
  • N is the number of predicted frames
  • M is the number of preset frames.
  • FIG 3 it is a schematic diagram of a lane line map obtained by searching for associated lane lines in one embodiment.
  • the lane line map includes three associated lane lines A-C, A-B, and A-D. Point A is the starting point of the lane line, which is the same as the above-mentioned point O. means the same.
  • Each associated lane line is uniformly sampled into N+M points, that is, each associated lane line is represented by a uniformly sampled point, thereby obtaining a lane line point set.
  • the map matrix is a K ⁇ (N+M) ⁇ 2-dimensional matrix, where K represents the number of associated lane lines.
  • N represents the number of predicted frames
  • M represents the number of preset frames
  • (N+M) ⁇ 2 represents the position of the N+M points of a certain associated lane line in the world coordinate system.
  • the associated lane lines of the obstacles to be predicted are searched in the map data according to the preset frame position data, the associated lane lines are sampled to obtain a lane line point set, and the lane line point set is converted into the to-be-predicted point set
  • the map matrix corresponding to the obstacle is obtained to obtain a map matrix that can distinguish the data in the direction of the abscissa and the direction of the ordinate. efficiency.
  • the steps of inputting the target matrix into the time information model, and obtaining the first feature matrix corresponding to the position matrix and the second feature matrix corresponding to the map matrix include:
  • Step 402 input the target matrix into the one-dimensional convolutional neural network model, and perform feature extraction in multiple direction dimensions on the position matrix and the map matrix in the target matrix respectively through the one-dimensional convolutional neural network model.
  • Step 404 Obtain a first feature matrix corresponding to the location matrix and a second feature matrix corresponding to the map matrix according to the extracted features.
  • the temporal information model may be a one-dimensional convolutional neural network model.
  • the one-dimensional convolutional neural network model can be a convolutional neural network model with a convolution kernel size of 1 and a stride of 1, so that the abscissa data and the ordinate data are processed separately without interfering with each other.
  • the one-dimensional convolutional neural network model is used to extract the features of the position matrix and the map matrix in the target matrix in multiple directions respectively. Since the position matrix and the map matrix both include the data in the abscissa direction and the data in the ordinate direction, many A direction dimension refers to the abscissa direction and the ordinate direction.
  • the first feature matrix corresponding to the position matrix may be obtained according to the features of the extracted position matrix, and the second feature matrix corresponding to the map matrix may be obtained according to the features of the extracted map matrix.
  • the number of channels corresponding to the features of the extracted target feature matrix is C
  • the first feature matrix is a C ⁇ 2-dimensional matrix, where 2 represents the x-direction dimension and the y-direction dimension.
  • the number of channels corresponding to the features of the extracted map matrix is also C
  • the second feature matrix is a K ⁇ C ⁇ 2-dimensional matrix, where K represents the number of lane lines in the map matrix, and 2 represents the x-direction dimension and the y-direction dimension.
  • the above-mentioned channel data may be the number of channels of the last convolutional layer of the one-dimensional convolutional neural network model.
  • the time information of the target matrix is extracted by the one-dimensional convolutional neural network model
  • the network structure of the one-dimensional convolutional neural network model is small, and the time information of multiple coordinate directions can be processed separately, effectively The calculation amount of the model is reduced, and the extraction efficiency of time information is improved.
  • the first feature matrix and the second feature matrix are integrated with spatial information, and the step of obtaining the spatial feature matrix includes:
  • Step 502 compare the first feature matrix with the second feature matrix to obtain the similarity.
  • Step 504 Calculate a third feature matrix corresponding to the second feature matrix according to the similarity.
  • Step 506 Combine the third feature matrix with the first feature matrix to obtain a spatial feature matrix.
  • the first feature matrix includes time information embodied by the position data of the obstacle to be predicted.
  • the second feature matrix includes time information embodied by the associated lane lines of the obstacles to be predicted.
  • the similarity refers to the similarity between the trajectory generated by the preset frame position data of the obstacle to be predicted and each associated lane line in the second feature matrix.
  • the computer device may calculate the similarity between the first feature matrix and the second feature matrix by multiplying the transpose of the first feature matrix and the second feature matrix to obtain a similarity vector.
  • the similarity vector includes a similarity vector between the trajectory generated by the preset frame position data of the obstacle to be predicted and each associated lane line in the second feature matrix.
  • all similarity vectors are normalized by the softmax normalization function to obtain a probability vector.
  • the probability vector includes the similarity between the trajectory generated by the preset frame position data of the obstacle to be predicted and each associated lane line in the second feature matrix.
  • the probability vector [0.1, 0.3, 0.6], it means that the similarity between the trajectory generated by the preset frame position data of the obstacle to be predicted and the associated lane lines A-C, A-B and A-D are 0.1, 0.3 and 0.1 respectively. 0.6.
  • Each associated lane line in the second feature matrix is multiplied by the corresponding probability value and added to obtain a third feature matrix corresponding to the second feature matrix.
  • the third feature matrix is a new map feature, which is a C ⁇ 2-dimensional matrix.
  • the third feature matrix is then combined with the first feature matrix to obtain a spatial feature matrix. Merging refers to concatenating the third feature matrix with the first feature matrix.
  • the spatial feature matrix is a 2C ⁇ 2-dimensional matrix.
  • the first feature matrix and the second feature matrix are compared to obtain the similarity
  • the third feature matrix corresponding to the second feature matrix is calculated according to the similarity
  • the third feature matrix is compared with the first feature matrix. Merge to get the spatial feature matrix.
  • the relationship between the preset frame position data of the obstacle to be predicted and the associated lane line can be obtained, and spatial information can be obtained, thereby further improving the accuracy of the trajectory prediction.
  • obtaining the preset frame position data of the obstacle to be predicted includes: obtaining the preset frame point cloud data of the obstacle to be predicted; inputting the preset frame point cloud data into the target detection model, and locating the to-be-predicted obstacle.
  • the position information corresponding to the obstacle in each frame is predicted, and the preset frame position data of the obstacle to be predicted is obtained according to the position information corresponding to the obstacle to be predicted in the preset frame.
  • the preset frame point cloud data refers to the historical continuous multi-frame point cloud data including the current frame point cloud data.
  • the point cloud data refers to the data that the sensor records the scanned surrounding environment information in the form of a point cloud.
  • the surrounding environment information includes the obstacles to be predicted in the surrounding environment of the vehicle, and there can be multiple obstacles to be predicted.
  • the point cloud data may specifically include three-dimensional coordinates of each point, laser reflection intensity, color information, and the like. The three-dimensional coordinates are used to represent the position information of the obstacle surface to be predicted in the surrounding environment.
  • the computer equipment inputs the preset frame point cloud data into the target detection model respectively, determines the three-dimensional bounding box corresponding to each frame of the obstacle to be predicted, and obtains the preset frame position data of the obstacle to be predicted.
  • the target detection model can be any of the target detection models such as PointNet, PointPillar, PolarNet, Semantic Segment Models (semantic segmentation model).
  • the three-dimensional bounding box includes the center point coordinates, size, orientation, etc. of each obstacle to be predicted. The coordinates of the center point represent the position information of the obstacle to be predicted.
  • the position of the obstacle to be predicted in each frame can be obtained accurately and quickly, which is beneficial to improve the accuracy of subsequent trajectory prediction.
  • a trajectory prediction device based on time and space learning including: a data acquisition module 602 , a matrix generation module 604 , a time information extraction module 606 , and a spatial information integration module 608 and a trajectory prediction module 610, where:
  • the data acquisition module 602 is configured to acquire preset frame position data and map data of obstacles to be predicted.
  • the matrix generation module 604 is configured to generate a target matrix according to the preset frame position data and the map data, and the target matrix includes a position matrix corresponding to the preset frame position data and a map matrix corresponding to the obstacle to be predicted.
  • the time information extraction module 606 is configured to input the target matrix into the time information model to obtain the first feature matrix corresponding to the position matrix and the second feature matrix corresponding to the map matrix.
  • the spatial information integration module 608 is configured to perform spatial information integration of the first feature matrix and the second feature matrix to obtain a spatial feature matrix.
  • the trajectory prediction module 610 is configured to input the spatial feature matrix into the trajectory prediction model to obtain the target trajectory of the obstacle to be predicted.
  • the time information extraction module 606 is further configured to input the target matrix into the one-dimensional convolutional neural network model, and perform multiple operations on the position matrix and the map matrix in the target matrix through the one-dimensional convolutional neural network model. Feature extraction in the coordinate direction; according to the extracted features, a first feature matrix corresponding to the position matrix and a second feature matrix corresponding to the map matrix are obtained.
  • the spatial information integration module 608 is further configured to compare the first feature matrix and the second feature matrix to obtain a similarity; calculate a third feature matrix corresponding to the second feature matrix according to the similarity; The three feature matrices are combined with the first feature matrix to obtain a spatial feature matrix.
  • the matrix generation module 604 is further configured to convert the preset frame position data of the obstacle to be predicted into a position matrix; determine the map matrix corresponding to the obstacle to be predicted according to the preset frame position data and the map data; The position matrix is merged with the map matrix to obtain the target matrix.
  • the matrix generation module 604 is further configured to search the map data for the associated lane line of the obstacle to be predicted according to the preset frame position data; perform sampling processing on the associated lane line to obtain a lane line point set; The line point set is converted into a map matrix corresponding to the obstacles to be predicted.
  • the data acquisition module 602 is further configured to acquire preset frame point cloud data of the obstacle to be predicted; input the preset frame point cloud data into the target detection model respectively, and locate the obstacle to be predicted in each frame According to the corresponding position information, the preset frame position data of the obstacle to be predicted is obtained.
  • Each module in the above-mentioned temporal and spatial learning-based trajectory prediction apparatus may be implemented in whole or in part by software, hardware, and combinations thereof.
  • the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
  • a computer device is provided, the internal structure of which can be shown in FIG. 7 .
  • the computer device includes a processor, memory, a communication interface, and a database connected by a system bus.
  • the processor of the computer device is used to provide computing and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium, an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions and a database.
  • the internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer device is used for storing data of a trajectory prediction method based on time and space learning.
  • the communication interface of the computer device is used to connect and communicate with an external terminal.
  • the computer-readable instructions when executed by the processor, implement a temporal and spatial learning-based trajectory prediction method.
  • FIG. 7 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device comprising a memory and one or more processors, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, makes the one or more processors execute the above methods to implement steps in the example.
  • One or more computer storage media storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the steps in each of the foregoing method embodiments.
  • the computer storage medium is a readable storage medium, and the readable storage medium may be non-volatile or volatile.
  • a vehicle in one of the embodiments, the vehicle may specifically include an automatic driving vehicle, and the vehicle includes the above-mentioned computer device, which can execute the steps in the above-mentioned embodiment of the trajectory prediction method based on time and space learning.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种基于时间与空间学习的轨迹预测方法，包括：获取待预测障碍物的预设帧位置数据，以及地图数据（202）；根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵（204）；将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵（206）；将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵（208）；及将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹（210）。

Description

基于时间与空间学习的轨迹预测方法、装置和计算机设备 技术领域
本申请涉及一种基于时间与空间学习的轨迹预测方法、装置、计算机设备、存储介质和交通工具。
背景技术
在自动驾驶过程中，预测周围环境中的障碍物在一定时间内的轨迹，是非常有必要的。通过对障碍物的未来轨迹进行预测，能够使自动驾驶车辆更早识别障碍物的意图，并根据障碍物意图来规划行驶路线以及行驶速度，从而避免碰撞，减少安全事故的发生。目前，可以通过基于深度学习的轨迹预测方法来进行轨迹预测，如将障碍物的历史轨迹信息与地图数据预处理为栅格图或向量化数据，进而用深度网络处理栅格图或者向量化数据。
障碍物的历史轨迹信息可以称为时间信息，障碍物的历史轨迹信息与地图数据的关系可以称为空间信息。由于时间信息与空间信息对障碍物的轨迹预测尤为重要，然而现有的基于深度学习的轨迹预测方法无法同时充分利用时间信息与空间信息，导致轨迹预测的准确性较低。
发明内容
根据本申请公开的各种实施例，提供一种基于时间与空间学习的轨迹预测方法、装置、计算机设备、存储介质和交通工具。
一种基于时间与空间学习的轨迹预测方法，包括：
获取待预测障碍物的预设帧位置数据，以及地图数据；
根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
一种基于时间与空间学习的轨迹预测装置，包括：
数据获取模块，用于获取待预测障碍物的预设帧位置数据，以及地图数据；
矩阵生成模块，用于根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
时间信息提取模块，用于将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
空间信息整合模块，用于将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
轨迹预测模块，用于将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
一种计算机设备，包括存储器和一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述一个或多个处理器执行以下步骤：
获取待预测障碍物的预设帧位置数据，以及地图数据；
根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
一个或多个存储有计算机可读指令的计算机存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行以下步骤：
获取待预测障碍物的预设帧位置数据，以及地图数据；
根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
一种交通工具，包括执行上述基于时间和空间学习的轨迹预测方法的步骤。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为一个或多个实施例中基于时间与空间学习的轨迹预测方法的应用环境图。
图2为一个或多个实施例中基于时间与空间学习的轨迹预测方法的流程示意图。
图3为一个或多个实施例中搜索关联车道线得到的车道线图示意图。
图4为一个或多个实施例中将目标矩阵输入至时间信息模型中，得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵步骤的流程示意图。
图5为一个或多个实施例中将第一特征矩阵与第二特征矩阵进行空间信息整合，得到空间特征矩阵步骤的流程示意图。
图6为一个或多个实施例中基于时间与空间学习的轨迹预测装置的框图。
图7为一个或多个实施例中计算机设备的框图。
具体实施方式
为了使本申请的技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
需要说明的是，本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
本申请提供的基于时间与空间学习的轨迹预测方法，可以应用于如图1所示的应用环境中。车载传感器102与车载计算机设备104通过网络进行通信。车载传感器的数量可以为一个也可以为多个。车载计算机设备可以简称为计算机设备。车载传感器102将采集到的点云数据发送至计算机设备104，计算机设备104对点云数据进行目标检测，获取到待预测障碍物的预设帧位置数据，并获取预先存储的地图数据，根据预设帧位置数据和地图数据生成目标矩阵，目标矩阵包括预设帧位置数据对应的位置矩阵和待预测障碍物对应的地图矩阵，从而将目标矩阵输入至时间信息模型中，得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵，将第一特征矩阵与第二特征矩阵进行空间信息整合，得到空间特征矩阵；进而将空间特征矩阵输入至轨迹预测模型中，得到待预测障碍物的目标轨迹。车载传感器102可以但不限于是激光雷达、激光扫描仪、摄像头。车载计算机设备104可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑、便携式可穿戴设备，也可以用独立的服务 器或者是多个服务器组成的服务器集群来实现。
在其中一个实施例中，如图2所示，提供了一种基于时间与空间学习的轨迹预测方法，以该方法应用于图1中的计算机设备为例进行说明，包括以下步骤：
步骤202，获取待预测障碍物的预设帧位置数据，以及地图数据。
待预测障碍物是指无人驾驶车辆在行驶过程中，其周围的动态障碍物。待预测障碍物可以包括行人、车辆等。预设帧位置数据是指待预测障碍物在历史连续多帧的位置，包括当前帧位置。
地图数据是指计算机设备中预先存储的高精度地图。精度地图中含有丰富、细致的道路交通信息元素。高精度地图不仅有高精度的坐标，同时还包括准确的道路形状，并且还包括每个车道的坡度、曲率、航向、高程、侧倾的数据等。高精度地图不仅会描绘道路，也会描绘出每条道路上存在的车道线。
无人驾驶车辆在驾驶的过程中，安装在车辆上的车载传感器可以将采集到的点云数据发送至计算机设备。计算机设备可以帧为单位保存点云数据，并记录每帧点云数据的数据采集时间等信息。其中，车载传感器可以是激光雷达、激光扫描仪、摄像头等。计算机设备可以实时进行轨迹预测，针对当前帧，计算机设备可以获取当前帧点云数据在内的预设帧点云数据，并对预设帧点云数据进行目标检测，确定每一帧中，待预测障碍物在世界坐标系中的位置，从而得到待预测障碍物的预设帧位置数据。预设帧数可以是预先设置的，同样的，预测帧数也可以是预先设置的，预测帧数是指轨迹预测得到的目标轨迹对应的帧数。例如，激光雷达的频率为10Hz，需要通过根据无人驾驶车辆在2s内的轨迹数据来预测3s内的目标轨迹，预设帧数为2*10＝20帧，预测帧数为3*10＝30帧。每一帧中，待预测障碍物在世界坐标系中的位置可以用(x,y)表示，因此，预设帧位置数据中包括横坐标方向(x方向)的数据以及纵坐标方向(y方向)的数据。
步骤204，根据预设帧位置数据和地图数据生成目标矩阵，目标矩阵包括预设帧位置数据对应的位置矩阵和待预测障碍物对应的地图矩阵。
目标矩阵是指将预设帧数据和地图数据进行整合后，得到的一个矩阵。
具体的，计算机设备可以先将预设帧位置数据转换为对应的位置矩阵，位置矩阵中包括待预测障碍物在每一帧的位置。根据预设帧位置数据在地图数据中搜索待预测障碍物的关联车道线，关联车道线是指待预测障碍物未来可能行驶的车道。从而根据关联车道线得到待预测障碍物对应的地图矩阵。地图矩阵中包括待预测障碍物对应的多条车道线、每条关联车道线对应的车道线点的位置，进而将位置矩阵与地图矩阵合并成一个矩阵，得到目标矩阵。
步骤206，将目标矩阵输入至时间信息模型中，得到位置矩阵对应的第一特征矩阵和地 图矩阵对应的第二特征矩阵。
目标矩阵中包括预设帧位置数据对应的位置矩阵和待预测障碍物对应的地图矩阵。第一特征矩阵是指包括位置矩阵中潜藏的时间信息的矩阵，即包括待预测障碍物的位置数据所体现的时间信息。第二特征矩阵是指包括地图矩阵中潜藏的时间信息的矩阵，即包括待预测障碍物的关联车道线所体现的时间信息。
计算机设备中预先存储有时间信息模型，时间信息模型是通过大量的样本数据训练得到的。例如，时间信息模型可以是卷积神经网络模型。将目标矩阵输入至时间信息模型中，通过时间信息模型处理目标矩阵中位置矩阵和地图矩阵的帧数通道，从而学习帧数通道中潜藏的时间信息，即提取到位置矩阵以及地图矩阵中的时间特征，进而根据提取的时间特征分别得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵。
步骤208，将第一特征矩阵与第二特征矩阵进行空间信息整合，得到空间特征矩阵。
为了获取待预测障碍物的预设帧位置数据与车道线信息之间的关系，可以将位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵进行空间信息整合。具体的，计算机设备可以计算第一特征矩阵和第二特征矩阵之间的相似度，从而根据该相似度以及地图矩阵对应的第二特征矩阵计算得到一个新的地图特征，作为第三特征矩阵，进而将第三特征矩阵与第一特征矩阵进行合并，实现将新的地图特征与待预测障碍物的位置特征进行连接，得到空间特征矩阵。空间特征矩阵用于表示待预测障碍物的预设帧位置数据与车道线信息之间的关系。
步骤210，将空间特征矩阵输入至轨迹预测模型中，得到待预测障碍物的目标轨迹。
计算机设备中预先存储有轨迹预测模型，轨迹预测模型和上述的时间信息模型可以是通过相同的样本数据训练得到的。例如，轨迹预测模型可以是Encode-Decode(编解码)网络模型，具体可以是卷积神经网络模型。通过轨迹预测模型对空间特征矩阵进行轨迹预测，输出目标轨迹。目标轨迹可以是预测帧数对应的轨迹，预测帧数是指计算机设备需要预测的未来时间段所对应的帧数。例如，需要预测3s内的目标轨迹，则预测帧数为3*10＝30帧。
在其中一个实施例中，在通过样本数据训练时间信息模型和轨迹预测模型的过程中，可以利用反向传播算法，如SGD(Stochastic Gradient Descent，随机梯度下降法)、Adam(Adaptive Moment Estimation，自适应矩估计)算法等优化方法等对上述模型进行训练，得到模型参数，将时间信息模型和轨迹预测模型与对应的模型参数进行存储，得到已训练的时间信息模型和已训练的轨迹预测模型。
在本实施例中，获取待预测障碍物的预设帧位置数据，以及地图数据，根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵，能够得到满足模型输入要求的数据，还能够减少 矩阵的数量，便于后续进行时间信息以及空间信息的整合。将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵，将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵，将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。能够充分利用待预测障碍物在预设帧内的时间信息和空间信息，提高了轨迹预测结果的准确性。
在其中一个实施例中，位置矩阵为标记有预设帧位置数据对应的横坐标方向以及纵坐标方向的矩阵，地图矩阵为标记有预设帧位置数据对应的关联车道线的横坐标方向以及纵坐标方向的矩阵，时间信息模型和轨迹预测模型均为一维卷积神经网络模型，时间信息模型和轨迹预测模型可以是同一类型的卷积神经网络模型，也可以是不同类型的卷积神经网络模型。
计算机设备可以在位置矩阵中标记预设帧位置数据对应的横坐标方向以及纵坐标方向，以及在地图矩阵中标记与预设帧位置数据对应的关联车道线的横坐标方向以及纵坐标方向。具体的，可以将位置矩阵中待预测障碍物在每一帧的位置的横坐标数据进行横坐标方向标记，将纵坐标数据进行纵坐标方向标记，用于区分横坐标方向的数据以及纵坐标方向的数据。当预设帧数为M时，位置矩阵可以表示为M×2维矩阵[[x_{-M+1},y_{-M+1}],[x_{-M+2},y_{-M+2}],...,[x_{0},y_{0}]，其中，[x_{-M+1},y_{-M+1}]为历史第M-1帧待预测障碍物在世界坐标系中的位置，[x_{0},y_{0}]表示当前帧待预测障碍物在世界坐标系中的位置，2表示x坐标方向和y坐标方向。地图矩阵中包括待预测障碍物对应的多条关联车道线、每条关联车道线对应的车道线点的位置，可以将车道线点的位置的横坐标数据进行横坐标方向标记，将纵坐标数据进行纵坐标方向标记，用于区分横坐标方向的数据以及纵坐标方向的数据。地图矩阵为K×(N+M)×2维矩阵，其中，K表示关联车道线的条数。N表示预测帧数，M表示预设帧数，(N+M)×2表示某条关联车道线的N+M个点在世界坐标系中的位置。
时间信息模型和轨迹预测模型均可以是卷积核大小(kernel size)为1，步长(stride)为1的卷积神经网络模型，从而将横坐标方向的数据和纵坐标方向的数据单独处理，互不干扰，还能够减小计算量，计算速度更快。
在其中一个实施例中，根据预设帧位置数据和地图数据生成目标矩阵包括：将待预测障碍物的预设帧位置数据转换为位置矩阵；根据预设帧位置数据以及地图数据确定待预测障碍物对应的地图矩阵；将位置矩阵与地图矩阵进行合并，得到目标矩阵。
预设帧位置数据包括待预测障碍物在各帧的位置坐标。位置矩阵可以为标记有预设帧位置数据对应的横坐标方向以及纵坐标方向的矩阵。根据预设帧位置数据在地图数据中搜索待预测障碍物的关联车道线，关联车道线是指待预测障碍物在初始帧位置后可能行驶的车道线。 在搜到到关联车道线后，可以将关联车道线采样为多个点，从而得到车道线点集，将车道线点集转换为地图矩阵。地图矩阵中标记有与预设帧位置数据对应的关联车道线的横坐标方向以及纵坐标方向。将位置矩阵与地图矩阵进行合并，得到目标矩阵。
在本实施例中，将待预测障碍物的预设帧位置数据转换为位置矩阵，根据预设帧位置数据以及地图数据确定待预测障碍物对应的地图矩阵，得到可区分横坐标方向的数据以及纵坐标方向的数据的位置矩阵，以及地图矩阵，有利于后续将横坐标方向的数据以及纵坐标方向的数据分开处理，减小了计算量，计算速度也更快。将位置矩阵与地图矩阵进行合并，得到目标矩阵，能够减少矩阵的数量，便于后续进行时间信息以及空间信息的整合。
进一步的，根据预设帧位置数据以及地图数据确定待预测障碍物对应的地图矩阵包括：根据预设帧位置数据在地图数据中搜索待预测障碍物的关联车道线；对关联车道线进行采样处理，得到车道线点集；将车道线点集转换为待预测障碍物对应的地图矩阵。
通过预设帧位置数据的初始帧位置数据在地图数据中搜索距离最近的车道线点，用O表示。例如，可以采用KNN(K-Nearest Neighbor，K近邻)方法来搜索车道线点O。以O为起点向待预测障碍物的行驶方向继续搜索车道线，根据搜索到的关联车道线生成车道线图。搜索的关联车道线长度可以为V*T*(N+M)，其中，V表示待预测障碍物在预设帧内的平均速度，可以根据预设帧位置数据计算得到，T表示相邻帧位置数据之间的时间间隔，如100毫秒，N为预测帧数，M为预设帧数。如图3所示，为其中一个实施例中搜索关联车道线得到的车道线图示意图，车道线图中包括三条关联车道线A-C、A-B和A-D，A点为车道线的起点，与上述O点的含义相同。分别将每条关联车道线均匀采样为N+M个点，即用均匀采样后的点表示每条关联车道线，从而得到车道线点集。将车道线点集转换为待预测障碍物对应的地图矩阵，地图矩阵为K×(N+M)×2维矩阵，其中，K表示关联车道线条数。N表示预测帧数，M表示预设帧数，(N+M)×2表示某条关联车道线的N+M个点在世界坐标系中的位置。
在本实施例中，通过根据预设帧位置数据在地图数据中搜索待预测障碍物的关联车道线，对关联车道线进行采样处理，得到车道线点集，将车道线点集转换为待预测障碍物对应的地图矩阵，得到可区分横坐标方向数据以及纵坐标方向数据的地图矩阵，有利于后续将横坐标方向的数据以及纵坐标方向的数据分开处理，减小了计算量，提高了计算效率。
在其中一个实施例中，如图4所示，将目标矩阵输入至时间信息模型中，得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵的步骤包括：
步骤402，将目标矩阵输入至一维卷积神经网络模型中，通过一维卷积神经网络模型分别对目标矩阵中位置矩阵和地图矩阵进行多个方向维度的特征提取。
步骤404，根据提取的特征得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵。
时间信息模型可以是一维卷积神经网络模型。一维卷积神经网络模型可以是卷积核大小(kernel size)为1，步长(stride)为1的卷积神经网络模型，从而使得横坐标数据和纵坐标数据单独处理，互不干扰。
一维卷积神经网络模型用于分别对目标矩阵中位置矩阵和地图矩阵进行多个方向维度的特征提取，由于位置矩阵和地图矩阵中均包括横坐标方向的数据与纵坐标方向的数据，多个方向维度是指横坐标方向以及纵坐标方向。可以根据提取的位置矩阵的特征得到位置矩阵对应的第一特征矩阵，根据提取的地图矩阵的特征得到地图矩阵对应的第二特征矩阵。提取的目标特征矩阵的特征对应的通道数目为C，第一特征矩阵为C×2维矩阵，其中，2表示x方向维度和y方向维度。提取的地图矩阵的特征对应的通道数目也为C，第二特征矩阵为K×C×2维矩阵，其中，K表示地图矩阵中车道线的条数，2表示x方向维度和y方向维度。进一步的，上述通道数据可以是一维卷积神经网络模型的最后一层卷积层的通道数目。
在本实施例中，通过一维卷积神经网络模型对目标矩阵进行时间信息提取，一维卷积神经网络模型的网络结构较小，且能够对多个坐标方向的时间信息进行单独处理，有效减小了模型的计算量，提升了时间信息的提取效率。
在其中一个实施例中，如图5所示，将第一特征矩阵与第二特征矩阵进行空间信息整合，得到空间特征矩阵的步骤包括：
步骤502，将第一特征矩阵和第二特征矩阵进行比对，得到相似度。
步骤504，根据相似度计算第二特征矩阵对应的第三特征矩阵。
步骤506，将第三特征矩阵与第一特征矩阵进行合并，得到空间特征矩阵。
第一特征矩阵包括待预测障碍物的位置数据所体现的时间信息。第二特征矩阵包括待预测障碍物的关联车道线所体现的时间信息。相似度是指待预测障碍物的预设帧位置数据所生成的轨迹与第二特征矩阵中每一条关联车道线的相似度。
计算机设备计算第一特征矩阵和第二特征矩阵之间的相似度的方式可以是将第一特征矩阵与第二特征矩阵的转置进行相乘处理，得到相似性向量。相似性向量包括待预测障碍物的预设帧位置数据所生成的轨迹与第二特征矩阵中每一条关联车道线的相似性向量。从而利用softmax归一化函数对所有的相似性向量进行归一化处理，得到概率向量。概率向量包括待预测障碍物的预设帧位置数据所生成的轨迹与第二特征矩阵中每一条关联车道线的相似度。以图3为例，概率向量＝[0.1,0.3,0.6]，则表示待预测障碍物预设帧位置数据所生成的轨迹与关联车道线A-C、A-B和A-D的相似度分别为0.1、0.3和0.6。
将第二特征矩阵中每条关联车道线与对应的概率值相乘，并相加，得到第二特征矩阵对应的第三特征矩阵。第三特征矩阵为一个新的地图特征，为一个C×2维矩阵。再将第三特征矩阵与第一特征矩阵进行合并，得到空间特征矩阵。合并是指将第三特征矩阵与第一特征矩阵连接在一起。空间特征矩阵为2C×2维矩阵。
在本实施例中，将第一特征矩阵和第二特征矩阵进行比对，得到相似度，根据相似度计算第二特征矩阵对应的第三特征矩阵，将第三特征矩阵与第一特征矩阵进行合并，得到空间特征矩阵。能够获取到待预测障碍物的预设帧位置数据与关联车道线之间的关系，得到空间信息，由此可进一步提高轨迹预测的准确性。
在其中一个实施例中，获取待预测障碍物的预设帧位置数据包括：获取待预测障碍物的预设帧点云数据；将预设帧点云数据分别输入至目标检测模型中，定位待预测障碍物在各帧对应的位置信息，根据待预测障碍物在预设帧对应的位置信息得到待预测障碍物的预设帧位置数据。
预设帧点云数据是指包括当前帧点云数据在内的历史连续多帧点云数据。点云数据是指传感器将扫描到的周围环境信息以点云形式记录的数据，周围环境信息中包括车辆周围环境中的待预测障碍物，待预测障碍物可以为多个。点云数据具体可以包括各点的三维坐标、激光反射强度、颜色信息等。三维坐标用于表示周围环境中待预测障碍物表面的位置信息。
计算机设备将预设帧点云数据分别输入至目标检测模型中，确定待预测障碍物在各帧对应的三维包围框，得到待预测障碍物的预设帧位置数据。目标检测模型可以是PointNet、PointPillar、PolarNet、Semantic Segment Models(语义分割模型)等目标检测模型中的任意一种。三维包围框中包括每个待预测障碍物的中心点坐标、大小、朝向等。中心点坐标表示待预测障碍物的位置信息。
在本实施例中，通过目标检测模型中对预设帧点云数据进行目标检测，能够准确、快速得到待预测障碍物在每一帧的位置，有利于提高后续轨迹预测的准确性。
在其中一个实施例中，如图6所示，提供了一种基于时间与空间学习的轨迹预测装置，包括：数据获取模块602、矩阵生成模块604、时间信息提取模块606、空间信息整合模块608和轨迹预测模块610，其中：
数据获取模块602，用于获取待预测障碍物的预设帧位置数据，以及地图数据。
矩阵生成模块604，用于根据预设帧位置数据和地图数据生成目标矩阵，目标矩阵包括预设帧位置数据对应的位置矩阵和待预测障碍物对应的地图矩阵。
时间信息提取模块606，用于将目标矩阵输入至时间信息模型中，得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵。
空间信息整合模块608，用于将第一特征矩阵与第二特征矩阵进行空间信息整合，得到空间特征矩阵。
轨迹预测模块610，用于将空间特征矩阵输入至轨迹预测模型中，得到待预测障碍物的目标轨迹。
在其中一个实施例中，时间信息提取模块606还用于将目标矩阵输入至一维卷积神经网络模型中，通过一维卷积神经网络模型分别对目标矩阵中位置矩阵和地图矩阵进行多个坐标方向的特征提取；根据提取的特征得到位置矩阵对应的第一特征矩阵和地图矩阵对应的第二特征矩阵。
在其中一个实施例中，空间信息整合模块608还用于将第一特征矩阵和第二特征矩阵进行比对，得到相似度；根据相似度计算第二特征矩阵对应的第三特征矩阵；将第三特征矩阵与第一特征矩阵进行合并，得到空间特征矩阵。
在其中一个实施例中，矩阵生成模块604还用于将待预测障碍物的预设帧位置数据转换为位置矩阵；根据预设帧位置数据以及地图数据确定待预测障碍物对应的地图矩阵；将位置矩阵与地图矩阵进行合并，得到目标矩阵。
在其中一个实施例中，矩阵生成模块604还用于根据预设帧位置数据在地图数据中搜索待预测障碍物的关联车道线；对关联车道线进行采样处理，得到车道线点集；将车道线点集转换为待预测障碍物对应的地图矩阵。
在其中一个实施例中，数据获取模块602还用于获取待预测障碍物的预设帧点云数据；将预设帧点云数据分别输入至目标检测模型中，定位待预测障碍物在各帧对应的位置信息，得到待预测障碍物的预设帧位置数据。
关于基于时间与空间学习的轨迹预测装置的具体限定可以参见上文中对于基于时间与空间学习的轨迹预测方法的限定，在此不再赘述。上述基于时间与空间学习的轨迹预测装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在其中一个实施例中，提供了一种计算机设备，其内部结构图可以如图7所示。该计算机设备包括通过系统总线连接的处理器、存储器、通信接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储一种基于时间与空间学习的轨迹预测方法的数据。该计算机设备的通信接口用于与外部的 终端连接通信。该计算机可读指令被处理器执行时以实现一种基于时间与空间学习的轨迹预测方法。
本领域技术人员可以理解，图7中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
一种计算机设备，包括存储器及一个或多个处理器，存储器中储存有计算机可读指令，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述各个方法实施例中的步骤。
一个或多个存储有计算机可读指令的计算机存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述各个方法实施例中的步骤。
其中，该计算机存储介质为可读存储介质，可读存储介质可以是非易失性，也可以是易失性的。
在其中一个实施例中，提供了一种交通工具，该交通工具具体可以包括自动驾驶车辆，交通工具包括上述计算机设备，可以执行上述基于时间和空间学习的轨迹预测方法实施例中的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不 脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种基于时间与空间学习的轨迹预测方法，包括：
    获取待预测障碍物的预设帧位置数据，以及地图数据；
    根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
    将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
    将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
    将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
  2. 根据权利要求1所述的方法，其特征在于，所述将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵包括：
    将所述目标矩阵输入至一维卷积神经网络模型中，通过所述一维卷积神经网络模型分别对所述目标矩阵中位置矩阵和地图矩阵进行多个坐标方向的特征提取；及
    根据提取的特征得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵。
  3. 根据权利要求1所述的方法，其特征在于，所述将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵包括：
    将所述第一特征矩阵和所述第二特征矩阵进行比对，得到相似度；
    根据所述相似度计算所述第二特征矩阵对应的第三特征矩阵；及
    将所述第三特征矩阵与所述第一特征矩阵进行合并，得到空间特征矩阵。
  4. 根据权利要求1所述的方法，其特征在于，所述获取待预测障碍物的预设帧位置数据包括：
    获取待预测障碍物的预设帧点云数据；及
    将所述预设帧点云数据分别输入至目标检测模型中，定位所述待预测障碍物在各帧对应的位置信息，得到所述待预测障碍物的预设帧位置数据。
  5. 根据权利要求1所述的方法，其特征在于，所述根据所述预设帧位置数据和所述地图数据生成目标矩阵包括：
    将待预测障碍物的预设帧位置数据转换为位置矩阵；
    根据所述预设帧位置数据以及所述地图数据确定所述待预测障碍物对应的地图矩阵； 及
    将所述位置矩阵与所述地图矩阵进行合并，得到目标矩阵。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述预设帧位置数据以及所述地图数据确定所述待预测障碍物对应的地图矩阵包括：
    根据所述预设帧位置数据在所述地图数据中搜索所述待预测障碍物的关联车道线；
    对所述关联车道线进行采样处理，得到车道线点集；及
    将所述车道线点集转换为所述待预测障碍物对应的地图矩阵。
  7. 一种基于时间与空间学习的轨迹预测装置，包括：
    数据获取模块，用于获取待预测障碍物的预设帧位置数据，以及地图数据；
    矩阵生成模块，用于根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
    时间信息提取模块，用于将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
    空间信息整合模块，用于将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
    轨迹预测模块，用于将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
  8. 根据权利要求7所述的装置，其特征在于，时间信息提取模块还用于将所述目标矩阵输入至一维卷积神经网络模型中，通过所述一维卷积神经网络模型分别对所述目标矩阵中位置矩阵和地图矩阵进行多个坐标方向的特征提取；及根据提取的特征得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵。
  9. 根据权利要求7所述的装置，其特征在于，空间信息整合模块还用于将所述第一特征矩阵和所述第二特征矩阵进行比对，得到相似度；根据所述相似度计算所述第二特征矩阵对应的第三特征矩阵；及将所述第三特征矩阵与所述第一特征矩阵进行合并，得到空间特征矩阵。
  10. 根据权利要求7所述的装置，其特征在于，矩阵生成模块还用于将待预测障碍物的预设帧位置数据转换为位置矩阵；根据所述预设帧位置数据以及所述地图数据确定所述待预测障碍物对应的地图矩阵；及将所述位置矩阵与所述地图矩阵进行合并，得到目标矩阵。
  11. 一种计算机设备，包括存储器及一个或多个处理器，所述存储器中存储有计算机 可读指令，所述计算机可读指令被所述一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
    获取待预测障碍物的预设帧位置数据，以及地图数据；
    根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
    将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
    将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
    将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
  12. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：将所述目标矩阵输入至一维卷积神经网络模型中，通过所述一维卷积神经网络模型分别对所述目标矩阵中位置矩阵和地图矩阵进行多个坐标方向的特征提取；及根据提取的特征得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵。
  13. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：将所述第一特征矩阵和所述第二特征矩阵进行比对，得到相似度；根据所述相似度计算所述第二特征矩阵对应的第三特征矩阵；及将所述第三特征矩阵与所述第一特征矩阵进行合并，得到空间特征矩阵。
  14. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：将待预测障碍物的预设帧位置数据转换为位置矩阵；根据所述预设帧位置数据以及所述地图数据确定所述待预测障碍物对应的地图矩阵；及将所述位置矩阵与所述地图矩阵进行合并，得到目标矩阵。
  15. 根据权利要求14所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：根据所述预设帧位置数据在所述地图数据中搜索所述待预测障碍物的关联车道线；对所述关联车道线进行采样处理，得到车道线点集；及将所述车道线点集转换为所述待预测障碍物对应的地图矩阵。
  16. 一个或多个存储有计算机可读指令的计算机存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
    获取待预测障碍物的预设帧位置数据，以及地图数据；
    根据所述预设帧位置数据和所述地图数据生成目标矩阵，所述目标矩阵包括所述预设 帧位置数据对应的位置矩阵和所述待预测障碍物对应的地图矩阵；
    将所述目标矩阵输入至时间信息模型中，得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵；
    将所述第一特征矩阵与所述第二特征矩阵进行空间信息整合，得到空间特征矩阵；及
    将所述空间特征矩阵输入至轨迹预测模型中，得到所述待预测障碍物的目标轨迹。
  17. 根据权利要求16所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：将所述目标矩阵输入至一维卷积神经网络模型中，通过所述一维卷积神经网络模型分别对所述目标矩阵中位置矩阵和地图矩阵进行多个坐标方向的特征提取；及根据提取的特征得到所述位置矩阵对应的第一特征矩阵和所述地图矩阵对应的第二特征矩阵。
  18. 根据权利要求16所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：将所述第一特征矩阵和所述第二特征矩阵进行比对，得到相似度；根据所述相似度计算所述第二特征矩阵对应的第三特征矩阵；及将所述第三特征矩阵与所述第一特征矩阵进行合并，得到空间特征矩阵。
  19. 根据权利要求16所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：将待预测障碍物的预设帧位置数据转换为位置矩阵；根据所述预设帧位置数据以及所述地图数据确定所述待预测障碍物对应的地图矩阵；及将所述位置矩阵与所述地图矩阵进行合并，得到目标矩阵。
  20. 一种交通工具，包括执行根据权利要求1-6中任一项所述的轨迹预测方法。
PCT/CN2021/090552 2021-04-28 2021-04-28 基于时间与空间学习的轨迹预测方法、装置和计算机设备 WO2022226837A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202180050155.4A CN115943400B (zh) 2021-04-28 2021-04-28 基于时间与空间学习的轨迹预测方法、装置和计算机设备
PCT/CN2021/090552 WO2022226837A1 (zh) 2021-04-28 2021-04-28 基于时间与空间学习的轨迹预测方法、装置和计算机设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/090552 WO2022226837A1 (zh) 2021-04-28 2021-04-28 基于时间与空间学习的轨迹预测方法、装置和计算机设备

Publications (1)

Publication Number Publication Date
WO2022226837A1 true WO2022226837A1 (zh) 2022-11-03

Family

ID=83847689

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/090552 WO2022226837A1 (zh) 2021-04-28 2021-04-28 基于时间与空间学习的轨迹预测方法、装置和计算机设备

Country Status (2)

Country Link
CN (1) CN115943400B (zh)
WO (1) WO2022226837A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016152000A1 (ja) * 2015-03-20 2016-09-29 株式会社デンソー 安全確認支援装置、安全確認支援方法
CN112348293A (zh) * 2021-01-07 2021-02-09 北京三快在线科技有限公司 一种障碍物的轨迹预测方法及装置
CN112364997A (zh) * 2020-12-08 2021-02-12 北京三快在线科技有限公司 一种障碍物的轨迹预测方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109583151B (zh) * 2019-02-20 2023-07-21 阿波罗智能技术(北京)有限公司 车辆的行驶轨迹预测方法及装置
CN109885066B (zh) * 2019-03-26 2021-08-24 北京经纬恒润科技股份有限公司 一种运动轨迹预测方法及装置
US11679764B2 (en) * 2019-06-28 2023-06-20 Baidu Usa Llc Method for autonomously driving a vehicle based on moving trails of obstacles surrounding the vehicle
CN112015847B (zh) * 2020-10-19 2021-03-02 北京三快在线科技有限公司 一种障碍物的轨迹预测方法、装置、存储介质及电子设备
CN112651557A (zh) * 2020-12-25 2021-04-13 际络科技(上海)有限公司 轨迹预测系统及方法、电子设备及可读存储介质
CN112651990B (zh) * 2020-12-25 2022-12-16 际络科技(上海)有限公司 运动轨迹预测方法及系统、电子设备及可读存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016152000A1 (ja) * 2015-03-20 2016-09-29 株式会社デンソー 安全確認支援装置、安全確認支援方法
CN112364997A (zh) * 2020-12-08 2021-02-12 北京三快在线科技有限公司 一种障碍物的轨迹预测方法及装置
CN112348293A (zh) * 2021-01-07 2021-02-09 北京三快在线科技有限公司 一种障碍物的轨迹预测方法及装置

Also Published As

Publication number Publication date
CN115943400B (zh) 2024-06-18
CN115943400A (zh) 2023-04-07

Similar Documents

Publication Publication Date Title
WO2022222095A1 (zh) 轨迹预测方法、装置、计算机设备和存储介质
CN111971574B (zh) 用于自动驾驶车辆的lidar定位的基于深度学习的特征提取
EP3714290B1 (en) Lidar localization using 3d cnn network for solution inference in autonomous driving vehicles
EP3714285B1 (en) Lidar localization using rnn and lstm for temporal smoothness in autonomous driving vehicles
WO2021134296A1 (zh) 障碍物检测方法、装置、计算机设备和存储介质
Mahaur et al. Road object detection: a comparative study of deep learning-based algorithms
KR20210043516A (ko) 궤적 계획 모델을 훈련하는 방법, 장치, 전자 기기, 저장 매체 및 프로그램
Wei et al. Vision-based lane-changing behavior detection using deep residual neural network
CN113239719B (zh) 基于异常信息识别的轨迹预测方法、装置和计算机设备
CN113587944B (zh) 准实时的车辆行驶路线生成方法、系统及设备
CN112001378B (zh) 基于特征空间的车道线处理方法、装置、车载终端和介质
CN113189989B (zh) 车辆意图预测方法、装置、设备及存储介质
US20220155096A1 (en) Processing sparse top-down input representations of an environment using neural networks
Al Mamun et al. Efficient lane marking detection using deep learning technique with differential and cross-entropy loss.
US20210383213A1 (en) Prediction device, prediction method, computer program product, and vehicle control system
WO2022006777A1 (zh) 对周围移动物体进行变道分类的方法、系统、计算机设备及存储介质
CN111414804B (zh) 识别框确定方法、装置、计算机设备、车辆和存储介质
JP2022035033A (ja) 情報処理システム、情報処理方法、プログラムおよび車両制御システム
WO2022226837A1 (zh) 基于时间与空间学习的轨迹预测方法、装置和计算机设备
JP7556142B2 (ja) 点群からの効率的な三次元物体検出
CN113836251B (zh) 一种认知地图构建方法、装置、设备及介质
US12125298B2 (en) Efficient three-dimensional object detection from point clouds
CN118072267A (zh) 一种车道线检测方法和装置
CN118840729A (zh) 车辆视觉定位方法、装置、电子设备及存储介质
CN116469074A (zh) 探测关于对象和/或自由空间的至少一部分的信息的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21938322

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21938322

Country of ref document: EP

Kind code of ref document: A1