CN115917559A - Trajectory prediction method, apparatus, computer device and storage medium - Google Patents
Trajectory prediction method, apparatus, computer device and storage medium Download PDFInfo
- Publication number
- CN115917559A CN115917559A CN202180050157.3A CN202180050157A CN115917559A CN 115917559 A CN115917559 A CN 115917559A CN 202180050157 A CN202180050157 A CN 202180050157A CN 115917559 A CN115917559 A CN 115917559A
- Authority
- CN
- China
- Prior art keywords
- matrix
- track
- obstacle
- predicted
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Remote Sensing (AREA)
- Traffic Control Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A trajectory prediction method, comprising: acquiring a motion trail (202) of an obstacle to be predicted; determining target map information (204) corresponding to the obstacle to be predicted according to the motion trail; converting the motion trajectory into a corresponding first trajectory matrix and the target map information into a corresponding first map matrix (206); and inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, performing feature extraction on the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track (208) of the obstacle to be detected.
Description
The application relates to a trajectory prediction method, a trajectory prediction device, a computer device, a storage medium and a vehicle.
In the automatic driving process, it is very necessary to predict the trajectory of an obstacle in the surrounding environment within a certain time. By predicting the future track of the obstacle, the automatic driving vehicle can recognize the intention of the obstacle earlier, and plan the driving route and the driving speed according to the intention of the obstacle, so that collision is avoided, and safety accidents are reduced. The traditional method is to extract the characteristics of the historical track information and map information of the obstacle through the existing track prediction model to realize track prediction, for example, the historical track information and the map information are preprocessed into a grid map or vectorization information, and then the grid map or the vectorization information is processed by a deep network.
Because the map information is particularly important for the track prediction of the obstacle, the existing track prediction model only can roughly consider the correlation between the map information and the track information of the obstacle, and cannot fully extract the deeper correlation between the map information and the obstacle information, so that the accuracy of the track prediction is low.
Disclosure of Invention
According to various embodiments disclosed herein, a trajectory prediction method, apparatus, computer device, storage medium, and vehicle are provided.
A trajectory prediction method, comprising:
acquiring a motion trail of an obstacle to be predicted;
determining target map information corresponding to the obstacle to be predicted according to the motion track;
converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; and
inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
A trajectory prediction device comprising:
the track acquisition module is used for acquiring the motion track of the obstacle to be predicted;
the map acquisition module is used for determining target map information corresponding to the barrier to be predicted according to the movement track;
the matrix conversion module is used for converting the motion track into a corresponding first track matrix and converting the target map information into a corresponding first map matrix; and
and the track prediction module is used for inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the processors, cause the one or more processors to perform the steps of:
acquiring a motion trail of an obstacle to be predicted;
determining target map information corresponding to the obstacle to be predicted according to the motion trail;
converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; and
inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
One or more computer storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
acquiring a motion track of an obstacle to be predicted;
determining target map information corresponding to the obstacle to be predicted according to the motion trail;
converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; and
inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
A vehicle comprises the steps of executing the trajectory prediction method.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description and drawings, and from the claims.
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a diagram of an application environment of a trajectory prediction method in one or more embodiments.
FIG. 2 is a flow diagram of a trajectory prediction method in one or more embodiments.
FIG. 3 is a block diagram of a trained trajectory prediction model in one or more embodiments.
Fig. 4 is a schematic flowchart of a step of performing embedding processing on the first track matrix and the first map matrix to obtain an object matrix in one or more embodiments.
FIG. 5 is a block diagram of a trajectory prediction device in one or more embodiments.
FIG. 6 is a block diagram of a computer device in one or more embodiments.
In order to make the technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It is noted that the terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The trajectory prediction method provided by the application can be applied to the application environment shown in fig. 1. The onboard sensors 102 communicate with the onboard computer device 104 over a network. The number of the vehicle-mounted sensors can be one or more. The on-board computer device may be referred to simply as a computer device. The vehicle-mounted sensor 102 sends the collected drive test data to the computer device 104, the computer device 104 performs detection, tracking and sampling processing on the drive test data to obtain a motion track of an obstacle to be predicted in the drive test data, determines target map information corresponding to the obstacle to be predicted according to the motion track, converts the motion track into a corresponding first track matrix, converts the target map information into a corresponding first map matrix, inputs the first track matrix and the first map matrix into a trained track prediction model, performs embedding processing on the first track matrix and the first map matrix to obtain a target matrix, performs feature extraction on the target matrix based on an attention mechanism to obtain output multi-head features, and performs regression processing on the output features to obtain a predicted track of the obstacle to be detected. The onboard sensors 102 may be, but are not limited to, lidar, laser scanners.
In one embodiment, as shown in fig. 2, a trajectory prediction method is provided, which is described by taking the example of the method applied to the computer device in fig. 1, and includes the following steps:
The obstacle to be predicted refers to a dynamic obstacle around the vehicle in the driving process of the vehicle. The obstacle to be predicted may include a pedestrian, a vehicle, and the like.
During driving of the vehicle, the sensors mounted on the vehicle may transmit the collected drive test data to the computer device. The computer equipment can store the drive test data by taking a frame as a unit and record the information such as data acquisition time of each frame of drive test data. The vehicle-mounted sensor can be a laser radar, a laser scanner, a camera and the like. The drive test data may be point cloud data or an image of the surroundings. And when the sensor is a laser radar or a laser scanner, the acquired point cloud data is sent to the computer equipment. When the sensor is a camera, the acquired surrounding environment image is sent to the computer equipment. The point cloud data refers to data recorded in a point cloud form by the sensor according to scanned surrounding environment information, the surrounding environment information includes obstacles to be predicted in the surrounding environment of the vehicle, and the number of the obstacles to be predicted can be multiple. The point cloud data may specifically include three-dimensional coordinates of each point, laser reflection intensity, color information, and the like. The three-dimensional coordinates are used to represent position information of the surface of the obstacle to be predicted in the surrounding environment. The surrounding image may be a panoramic image of the surroundings of the vehicle captured by a plurality of cameras.
And when the computer equipment acquires the drive test data in the preset time period, carrying out target detection and target tracking on the drive test data to obtain a motion track in the preset time period. For example, the preset time period may be 2s. The target detection means detecting obstacles in the drive test data and predicting the position and category of each obstacle. The target tracking means that under the condition that the position of an obstacle in an initial frame is known, the position of the obstacle in a subsequent frame is predicted, and the speed information of the obstacle to be predicted is determined. The trajectory information includes position information, speed, orientation, and the like of the obstacle to be predicted in each frame of drive test data. The position information refers to position coordinates of the obstacle to be predicted in world coordinates. Specifically, the computer device inputs the acquired drive test data into the corresponding target detection model, positions the position area where each obstacle to be predicted is located, and frames the position area with a surrounding frame to obtain a surrounding frame corresponding to each obstacle to be predicted. The surrounding frame comprises the coordinates, the size, the orientation and the like of the center point of each obstacle to be predicted. The coordinates of the center point of the bounding box represent the position information of the obstacle to be predicted. Different obstacles to be predicted can be accurately distinguished by identifying the bounding boxes corresponding to the obstacles to be predicted. The computer equipment can input a current frame bounding box corresponding to the obstacle to be predicted and a continuous multi-frame bounding box formed by a bounding box before the current frame into a pre-trained target tracking model to obtain the speed and the acceleration of the obstacle to be predicted of the current frame. And the computer equipment performs target detection and target tracking on each frame of drive test data to obtain the track information of the obstacle to be predicted in each frame. The computer equipment stores a high-precision map, and the high-precision map contains abundant and detailed road traffic information elements. The high-precision map not only has high-precision coordinates, but also includes accurate road shapes, and also includes data of the gradient, curvature, heading, elevation, roll of each lane, and the like. The high-precision map can not only depict roads, but also depict a plurality of lanes on one road, and can truly reflect the actual style of the roads. The computer equipment can sample and process the track information of the barrier to be predicted based on the high-precision map to obtain the track meeting the preset sampling condition, so that the movement track of the barrier to be predicted is obtained. The preset sampling conditions refer to a track of an intersection (junction) area, a track with changed curvature and speed, a track with lane change (lane change) and a track with cut in. The motion trail comprises a plurality of trail points, and each trail point comprises x-direction and y-direction coordinate values.
And 204, determining target map information corresponding to the obstacle to be predicted according to the motion trail.
The computer device searches the lane central lines corresponding to the movement tracks, and the number of the lane central lines can be multiple. And sampling the lane center line, and representing the lane center line by a plurality of sampled points. The sampled points may be referred to as location points. And obtaining target map information corresponding to the obstacle to be predicted according to the lane center line. The target map information may include lane centerlines corresponding to each motion trajectory, and each motion trajectory may correspond to a plurality of lane centerlines. The lane center line corresponding to each movement track may be referred to as a track map information, and a lane center line may be referred to as a track lane information, and thus, one track map information may include a plurality of track lane information. Each lane centerline includes a plurality of location points, each location point including x-direction and y-direction coordinate values.
Since the trajectory includes coordinate values in the x direction and the y direction, the x direction and the y direction may be separately calculated in order to increase the trajectory prediction speed. Specifically, the movement track may be converted into a first track matrix, and the target map information may be converted into a corresponding first map matrix. The first track matrix is in a format of N _1 × T1 × 2, where N _1 represents the number of tracks in a motion track, T1 represents the number of tracks in each track, and 2 represents the x and y coordinate directions. The second map matrix is in a matrix format of N _2 × T2 × 2, where N _2 represents the number of pieces of track lane information in the target map information, T2 represents the number of position points in each piece of track lane information, and 2 represents the x and y coordinate directions.
And 208, inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
The target matrix refers to a matrix that identifies the positional relationship of the obstacle to be predicted and the corresponding target map information.
The computer device stores a trained track prediction model in advance, and the trained track prediction model is a model based on a multi-head attention network. A multi-head attention network refers to a transformer network. The trained track prediction model is obtained by training a large amount of sample data. The trained trajectory prediction model may include an embedded network, a multi-head attention network, and a regression network. The embedded network may be formed by any one of the existing one-dimensional convolution networks, and is configured to perform embedding processing on the first trajectory matrix and the first map matrix, where the embedding processing may include performing feature extraction on the first trajectory matrix and the first map matrix, and performing position embedding on the extracted feature matrix to obtain an object matrix. Position embedding refers to identifying the position relationship between an obstacle to be predicted and corresponding target map information.
And taking the target matrix as the input of a multi-head attention network, and performing feature extraction on the target matrix through the multi-head attention network based on a multi-attention mechanism to obtain an output matrix. The output matrix is a matrix obtained by connecting matrices extracted by a plurality of attention heads. The multi-attention mechanism is a multi-head self-attention (multi-head self-attention) layer feature extraction mechanism in a multi-head attention network, can focus on the relation between an obstacle to be predicted and target map information in a target matrix from different positions, can obtain more abundant and comprehensive feature information, and can fully extract deeper correlation between the map information and the obstacle information. The regression network may be any one of the existing one-dimensional convolutional neural networks. And inputting the output matrix into a regression network, and performing prediction operation on the output matrix through the regression network to obtain a predicted track of the obstacle to be predicted. The predicted trajectory may be a trajectory of movement of the obstacle to be predicted over a period of time in the future, such as a trajectory of movement over 3s in the future.
In this embodiment, the movement track of the obstacle to be predicted is obtained, the target map information corresponding to the obstacle to be predicted is determined according to the movement track, the movement track is converted into the corresponding first track matrix, and the target map information is converted into the corresponding first map matrix, so that the movement track and the target map information meet the input requirement of the track prediction model. Inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected. Due to the fact that the multi-head attention mechanism in the track prediction model can focus on the relation between the to-be-predicted barrier and the target map information in the target matrix from different positions, richer and comprehensive characteristic information can be obtained, the deep correlation between the map information and the barrier information can be fully extracted, and the accuracy of track prediction is improved. In addition, the movement track and the target map information can be divided into information in the x direction and the y direction by converting the movement track into the corresponding first track matrix and converting the target map information into the corresponding first map matrix, so that the information in the x direction and the y direction can be independently operated, and the track prediction efficiency is improved.
In one embodiment, the obtaining of the movement locus of the obstacle to be predicted includes: acquiring drive test data, and sensing the drive test data to obtain track information of an obstacle to be predicted in the drive test data; and sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
The drive test data refers to environmental information around the autonomous vehicle acquired by a sensor during autonomous driving.
The computer equipment acquires the drive test data acquired by the sensor and carries out sensing processing on the drive test data, wherein the sensing processing refers to target detection and target tracking on the drive test data. The drive test data may be point cloud data or an image of the surroundings. When the drive test data is point cloud data, the point cloud data can be subjected to target detection through any one of target detection Models, such as PointNet, pointpilar, polarNet, semantic Segment Models and the like, so as to determine a three-dimensional enclosure frame corresponding to each obstacle to be predicted, including the center point coordinates, the size, the orientation and the like of each obstacle to be predicted. The center point coordinates represent position information of the obstacle to be predicted. When the road measurement data is a surrounding environment image, a two-dimensional bounding box corresponding to the obstacle to be predicted may be determined by performing target Detection on the surrounding environment image through any one of a target Detection model, such as a SSD (Single-Shot multi-box Detector direct multi-target Detection) model, a refine-direct multi-target Detection (fine-direct multi-target Detection) model, a mobile-based Single-Shot multi-box Detector (direct multi-target Detection based on a high-efficiency convolutional neural network for mobile-end visual application) model, a YOLO (unified real-time target Detection) model, and the like, including a center point coordinate, a size, an orientation, and the like of the obstacle to be predicted. The center point coordinates represent position information of the obstacle to be predicted.
In the target tracking process, the speed information of the obstacle to be detected in the subsequent frame can be predicted through any one of the traditional trackers such as a kalman Filter (KF for short) and an unscented kalman Filter (UKF for short), and the like, of the target tracking model. And carrying out target detection and target tracking on each frame of drive test data to obtain the track information of the obstacle to be predicted in each frame. Because the track information of the obstacle to be detected may be static or uniform motion, in order to improve the accuracy of track prediction, the track information may be sampled, and only the track which is not static or varies at a uniform speed is sampled. The computer equipment can perform sampling processing on the track information of the obstacle to be predicted according to preset sampling conditions to obtain the movement track of the obstacle to be predicted. The preset sampling conditions refer to a track of an intersection (junction) area, a track with changed curvature and speed, a track with lane change (lane change) and a track with cut in. The motion trail comprises a plurality of trail points, and each trail point comprises x-direction and y-direction coordinate values.
In this embodiment, sensing processing is performed on the drive test data to obtain trajectory information of the obstacle to be predicted in the drive test data, and sampling processing is performed on the trajectory information of the obstacle to be predicted according to a preset sampling condition to obtain a motion trajectory corresponding to the obstacle to be predicted. By sampling representative track information, the accuracy of track prediction can be effectively improved.
In one embodiment, determining target map information corresponding to an obstacle to be predicted according to the motion trail includes: determining a corresponding lane central line according to the motion track; and sampling the lane central line to obtain target map information corresponding to the obstacle to be predicted.
And determining the position of an initial track point of the movement track aiming at the movement track of the obstacle to be predicted, and determining a circle area with the radius of r by taking the position as the center of a circle, wherein r is 3m. The computer equipment determines lane central lines with intersections with the circle area based on the high-precision map, the number of the lane central lines with intersections can be multiple, and target map information corresponding to the movement track is obtained according to the multiple lane central lines with intersections. If the initial track point of the obstacle to be predicted is located closer to the lane boundary (lane change may occur), the lane center line of the obstacle to be predicted includes the lane center line where the initial track point is located and the lane center line of the lane to be changed. Because vehicles on the road run along lanes, map information beside the tracks is important for predicting the tracks, in order to improve the accuracy of track prediction, each lane central line corresponding to the motion track can be uniformly sampled into N points, namely each lane central line is represented by the sampled points, and each lane central line comprises N position points. The number of sampling points can be set according to the duration of motion estimation and the duration of the track to be predicted.
In the embodiment, the lane central line corresponding to the movement track is determined, the lane central line is sampled to obtain the target map information corresponding to the obstacle to be predicted, the target map information related to the movement track can be accurately obtained, and the accuracy of track prediction is improved.
In one embodiment, the trained trajectory prediction model includes a multi-head attention network, and the multi-head attention network includes a one-dimensional convolutional layer, and the one-dimensional convolutional layer is used for respectively performing feature extraction in an abscissa direction and feature extraction in an ordinate direction on the target matrix.
Fig. 3 is a schematic structural diagram of the trained trajectory prediction model. The trained trajectory prediction model comprises an embedded network, a multi-head attention network and a regression network which are connected in sequence. The multi-headed attention network is a transform network, and x N represents that the transform network comprises a plurality of multi-headed attention layers and a feedforward neural network layer, and an Add & Norm layer is arranged behind the multi-headed attention layers and the feedforward neural network layer.
The multi-head attention layer extracts the characteristics of the target matrix through a multi-head attention mechanism, and the multi-head attention mechanism can focus on track points at different positions in the target matrix. The calculation paths of the plurality of track points are input to the feedforward neural network layer, so that more matrix vectors in the multi-head attention network interact, and more complex relations can be learned. Because the path has no dependency relationship in the feedforward unit, the calculation paths of a plurality of trace points can be executed in parallel through the feedforward neural network layer, and the output characteristic is obtained.
Add is a residual error network, and a residual error structure can well solve the problem of information loss caused by deepening the layer number. Norm refers to Layer Normalization, and thus the Add & Norm unit is used to Add the inputs and outputs of the multi-head attention Layer, or feedforward neural network Layer, and perform Normalization. Layer Normalization is used to convert the input into data with a mean of 0 and a variance of 1, avoiding the input falling in the saturation region of the subsequent activation function.
The traditional transform network includes a Linear layer, while the transform network in the embodiment is an improved transform network, and the specific method is to replace the Linear layer in the traditional transform network with a one-dimensional convolutional layer. Therefore, the target matrix can be subjected to feature extraction in the abscissa direction and feature extraction in the ordinate direction through the one-dimensional convolution layer, the embedded network and the regression network of the transform network both adopt one-dimensional convolution networks, independent operation of data in the x direction and the y direction is achieved, the track prediction efficiency is effectively improved, and the track prediction accuracy is also improved.
In one embodiment, as shown in fig. 4, the embedding the first trajectory matrix and the first map matrix to obtain the target matrix includes:
And 404, combining the first characteristic matrix and the second characteristic matrix to obtain a combined matrix.
And 406, adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
The trained trajectory prediction model includes an embedding (embedding) network, which may be a one-dimensional convolution network, a multi-head attention network, and a regression network. The embedded network is used for converting the first track matrix and the first map matrix into a matrix format required by the multi-head attention network, and can be used for capturing the relation between track points in the first track matrix and position points in the first map matrix in a high-dimensional space. And respectively extracting features of the first track matrix and the first map matrix through the embedded network, and generating a first feature matrix corresponding to the first track matrix and a second feature matrix corresponding to the first map matrix according to the number of channels of the last layer of convolution layer of the embedded network. The number of channels in the last convolutional layer can be represented by dim 1. The first feature matrix may be represented as N _1 × dim1 × 2, where N _1 represents the number of tracks in the first feature matrix and 2 represents the x and y coordinate directions. The second feature matrix may be represented as N _2 × dim1 × 2, where N _2 represents the number of pieces of track lane information in the second feature matrix, and 2 represents the x and y coordinate directions.
And combining the first characteristic matrix and the second characteristic matrix in a second dimension through an embedded network to obtain a combined matrix, wherein the combined matrix is a four-dimensional matrix and can be represented as N _1 × dim2 × dim _1 × 2, and dim2 represents the total feature number of the second dimension after the first characteristic matrix and the second characteristic matrix are combined in the second dimension. dim2 may be preset so that the computer device merges the first feature matrix and the second feature matrix according to the preset value. In the merging process, traversing each obstacle to be predicted, and if the number of track lane information pieces +1 corresponding to the track of the obstacle to be predicted in the second feature matrix is greater than dim2, randomly selecting dim2-1 track lane information from the number of track lane information pieces in the second feature matrix and merging the track of the obstacle to be predicted in the first feature matrix in a second dimension; if the number of the track lane information +1 corresponding to the track of the obstacle to be predicted in the second feature matrix is smaller than dim2, a 0 matrix needs to be stacked in the second dimension, so that the total feature number of the merged second dimension is dim2.
The embedded network adds the feature parameter in the second dimension of the combination matrix, and the combination matrix after adding the feature parameter can be represented as N _1 × (1 + dim2) × dim _1 × 2, where 1 represents the added feature parameter, and the feature parameter can be any value. The characteristic parameters are used for collecting the information of the map and the obstacle to be predicted in proportion so as to perform subsequent track prediction.
Since the multi-head attention network does not process the position relation between the obstacle to be predicted and the map information, the position embedding (position embedding) process can be performed on the combined matrix added with the characteristic parameters to obtain the target matrix. The position relation between the barrier to be predicted and the map information in the matrix can be identified through position embedding, and the position embedding is used for making up the deficiency of the position information. The target matrix can be directly input into a multi-head attention network for feature extraction.
In this embodiment, feature extraction is performed on the first trajectory matrix and the first map matrix through the embedded network, and a first feature matrix corresponding to the first trajectory matrix and a second feature matrix corresponding to the first map matrix are obtained according to the number of channels of the last convolution layer of the embedded network, so that a matrix format required by the multi-head attention network can be obtained, and the method can be used for capturing a relationship between a trajectory point in the first trajectory matrix and a position point in the first map matrix in a high-dimensional space. The first characteristic matrix and the second characteristic matrix are combined to obtain a combined matrix, characteristic parameters are added into the combined matrix, information of a map and an obstacle to be predicted can be rapidly collected to perform subsequent track prediction, position embedding processing is performed on the combined matrix after the characteristic parameters are added, the defect of position information between the obstacle to be predicted and the map information in the multi-head attention network can be made up, and the accuracy of track prediction can be further improved.
In one embodiment, before obtaining the motion trajectory of the obstacle to be predicted, the method further includes: acquiring a training sample, wherein the training sample comprises track information of a target obstacle and sample map information corresponding to the target obstacle; converting the track information into a corresponding second track matrix, and converting the sample map information into a corresponding second map matrix; inputting the second track matrix and the second map matrix into a track prediction model to be trained, and outputting a future track of the target barrier; and calculating the model loss of the track prediction model to be trained according to the track information and the future track, and updating the model parameters of the track prediction model to be trained according to the model loss until the preset condition is met to obtain the trained track prediction model.
The training samples refer to sample data used for training a trajectory prediction model, and the training samples comprise trajectory information of a target obstacle and sample map information corresponding to the target obstacle. The target obstacle refers to a dynamic obstacle such as a vehicle, a pedestrian, and the like. Specifically, the computer device obtains historical drive test data acquired by the sensor, and senses the historical drive test data to obtain track information of the dynamic obstacle in the historical drive test data. The perception processing refers to target detection and target tracking, and is the same as the perception processing mode of the trajectory prediction model in the application process, and is not described herein again. Similarly, the computer device performs sampling processing on the track information of the dynamic obstacle according to a preset sampling condition to obtain a track sample set corresponding to the dynamic obstacle. The preset sampling condition may be a trajectory of an intersection (junction) area, a trajectory in which a curvature and a speed change are changed, a trajectory in which lane change occurs, and a trajectory in which a passing lane (cut in) is passed. The track sample set includes historical tracks of a plurality of dynamic obstacles, each historical track includes a plurality of track points, for example, each historical track may include 50 track points. Each trace point includes x-direction and y-direction coordinate values. And determining track lane information corresponding to the dynamic barrier according to each historical track in the track sample set to obtain a map sample set corresponding to the dynamic barrier. The track lane information sampling mode is the same as the track prediction model sampling mode in the application process, and details are not repeated here. And respectively selecting the track information and the sample map information corresponding to the target barrier in the track sample set and the map sample set to generate a training sample. The track sample set and the map sample set can be divided into a training sample, a test set and a verification set according to a preset proportion. For example, the preset ratio may be 3. The track sample set and the map sample set are divided into three sets, so that a model with the highest accuracy and the best generalization capability is selected.
The track prediction mode in the training process and the application process of the track prediction model is the same, namely, the track information in the training samples is converted into corresponding second track matrixes, the sample map information in the training samples is converted into corresponding second map matrixes, the second track matrixes and the second map matrixes are input into the track prediction model to be trained, and the future tracks of the target obstacles are output. Therefore, the model loss of the track prediction model to be trained is calculated according to the track information and the future track, and the model parameters are adjusted according to the model loss to obtain the trained track prediction model. For example, the model loss may be an existing loss function such as MSE mean square error loss, cross entropy loss, and the like, and the model parameters are adjusted by back propagation of the output of the loss function, since the model training process is an iterative training process and needs to pass through a plurality of epochs, 1 epoch represents that all training samples are used for training once, and each epoch outputs one model parameter. The model parameter with the highest accuracy can be determined through the verification set, namely, the future trajectory obtained by the model parameter output by which epoch is judged is more accurate, the specific judgment mode can be to judge whether the network loss value reaches a loss threshold or whether the iteration number reaches an iteration number threshold, if the network loss value reaches the loss threshold or the iteration number reaches the iteration number threshold, the model parameter output by the corresponding epoch can be used as the final model parameter, and the model is a trained trajectory prediction model. After the trained trajectory prediction model is obtained, the model prediction can be performed by using the test set to measure the performance of the model. If the performance of the model tested by the test set is poor, the model parameters of the model can be readjusted by using the training sample until the model parameters with the highest accuracy are obtained.
In this embodiment, a training sample is obtained, where the training sample includes trajectory information of a target obstacle and sample map information corresponding to the target obstacle, the trajectory information is converted into a corresponding second trajectory matrix, the sample map information is converted into a corresponding second map matrix, the second trajectory matrix and the second map matrix are input into a trajectory prediction model to be trained, a model loss of the trajectory prediction model to be trained is calculated, and a model parameter of the trajectory prediction model to be trained is updated according to the model loss, so as to obtain a trained trajectory prediction model. Because the multi-head attention mechanism in the track prediction model can focus on the relation between the target barrier and the sample map information in the target matrix from different positions, more abundant and comprehensive characteristic information can be obtained, deeper correlation between the map information and the barrier information can be fully extracted, and the accuracy of track prediction is improved.
In one embodiment, as shown in fig. 5, there is provided a trajectory prediction device, including: a trajectory acquisition module 502, a map acquisition module 504, a matrix conversion module 506, and a trajectory prediction module 508, wherein:
and a track acquiring module 502 for acquiring the motion track of the obstacle to be predicted.
And the map obtaining module 504 is configured to determine target map information corresponding to the obstacle to be predicted according to the movement trajectory.
The matrix conversion module 506 is configured to convert the motion trajectory into a corresponding first trajectory matrix, and convert the target map information into a corresponding first map matrix.
The trajectory prediction module 508 is configured to input the first trajectory matrix and the first map matrix into a trained trajectory prediction model, perform embedding processing on the first trajectory matrix and the first map matrix to obtain a target matrix, perform feature extraction on the target matrix based on a multi-head attention mechanism to obtain an output feature, and perform regression processing on the output feature to obtain a predicted trajectory of the obstacle to be detected.
In one embodiment, the trained trajectory prediction model includes a multi-head attention network, the multi-head attention network includes a one-dimensional convolutional layer, and the trajectory prediction module 508 is further configured to perform feature extraction in the abscissa direction and feature extraction in the ordinate direction on the target matrix according to the one-dimensional convolutional layer, respectively.
In one embodiment, the trajectory prediction module 508 is further configured to perform feature extraction on the first trajectory matrix and the first map matrix through an embedded network in the trained trajectory prediction model, obtain the number of channels of the last convolution layer embedded in the network, and obtain a first feature matrix corresponding to the first trajectory matrix and a second feature matrix corresponding to the first map matrix according to the number of channels; merging the first characteristic matrix and the second characteristic matrix to obtain a combined matrix; and adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
In one embodiment, the trajectory acquisition module 508 is further configured to acquire drive test data, and perform sensing processing on the drive test data to obtain trajectory information of an obstacle to be predicted in the drive test data; and sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
In one embodiment, the map obtaining module 504 is further configured to determine a corresponding lane center line according to the motion trajectory; and sampling the center line of the lane to obtain target map information corresponding to the obstacle to be predicted.
In one embodiment, the apparatus further comprises:
the sample acquisition module is used for acquiring training samples, and the training samples comprise track information of the target obstacle and sample map information corresponding to the target obstacle.
And the sample conversion module is used for converting the track information into a corresponding second track matrix and converting the sample map information into a corresponding second map matrix.
And the track operation module is used for inputting the second track matrix and the second map matrix into a track prediction model to be trained and outputting the future track of the target obstacle.
And the parameter updating module is used for calculating the model loss of the track prediction model to be trained according to the track information and the future track, and updating the model parameters of the track prediction model to be trained according to the model loss to obtain the trained track prediction model.
For the specific definition of the trajectory prediction device, reference may be made to the above definition of the trajectory prediction method, which is not described herein again. The modules in the trajectory prediction device may be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 6. The computer device includes a processor, a memory, a communication interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operating system and execution of computer-readable instructions in the non-volatile storage medium. The database of the computer device is used for storing data of a trajectory prediction method. The communication interface of the computer device is used for connecting and communicating with an external terminal. The computer readable instructions, when executed by a processor, implement a trajectory prediction method.
It will be appreciated by those skilled in the art that the configuration shown in fig. 6 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of the various method embodiments described above.
One or more computer storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps in the various method embodiments described above.
The computer storage medium is a readable storage medium, and the readable storage medium may be nonvolatile or volatile.
In one embodiment, a vehicle is provided, which may specifically include an autonomous vehicle, the vehicle including the computer device described above, and the steps of the trajectory prediction method embodiments described above may be performed.
Those skilled in the art will appreciate that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to computer readable instructions, which can be stored in a non-volatile computer readable storage medium, and when executed, the computer readable instructions can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (20)
- A trajectory prediction method, characterized in that the method comprises:acquiring a motion trail of an obstacle to be predicted;determining target map information corresponding to the obstacle to be predicted according to the motion track;converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; andinputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
- The method of claim 1, wherein the obtaining a motion trajectory of an obstacle to be predicted comprises:acquiring drive test data, and performing sensing processing on the drive test data to obtain track information of an obstacle to be predicted in the drive test data; andand sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
- The method according to claim 1, wherein the determining target map information corresponding to the obstacle to be predicted according to the motion trail comprises:determining a corresponding lane central line according to the motion track; andand sampling the lane center line to obtain target map information corresponding to the obstacle to be predicted.
- The method of claim 1, wherein the trained trajectory prediction model comprises a multi-head attention network, and wherein the multi-head attention network comprises one-dimensional convolutional layers for respectively performing feature extraction in an abscissa direction and feature extraction in an ordinate direction on the object matrix.
- The method of claim 1, wherein the embedding the first trajectory matrix and the first map matrix to obtain an object matrix comprises:respectively extracting features of the first track matrix and the first map matrix through an embedded network in the trained track prediction model, acquiring the number of channels of the last layer of convolution layer of the embedded network, and acquiring a first feature matrix corresponding to the first track matrix and a second feature matrix corresponding to the first map matrix according to the number of channels;merging the first characteristic matrix and the second characteristic matrix to obtain a combined matrix; andand adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
- The method according to any one of claims 1 to 5, characterized in that before said obtaining the motion trajectory of the obstacle to be predicted, the method further comprises:acquiring a training sample, wherein the training sample comprises track information of a target obstacle and sample map information corresponding to the target obstacle;converting the track information into a corresponding second track matrix, and converting the sample map information into a corresponding second map matrix;inputting the second trajectory matrix and the second map matrix into a trajectory prediction model to be trained, and outputting a future trajectory of the target obstacle; andcalculating the model loss of the to-be-trained track prediction model according to the track information and the future track, and updating the model parameters of the to-be-trained track prediction model according to the model loss to obtain the trained track prediction model.
- A trajectory prediction device comprising:the track acquisition module is used for acquiring the motion track of the obstacle to be predicted;the map acquisition module is used for determining target map information corresponding to the barrier to be predicted according to the movement track;the matrix conversion module is used for converting the motion track into a corresponding first track matrix and converting the target map information into a corresponding first map matrix; andthe track prediction module is used for inputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
- The apparatus of claim 7, wherein the trained trajectory prediction model comprises a multi-head attention network, the multi-head attention network comprises a one-dimensional convolutional layer, and the trajectory prediction module is further configured to perform feature extraction in an abscissa direction and feature extraction in an ordinate direction on the object matrix according to the one-dimensional convolutional layer, respectively.
- The apparatus according to claim 7, wherein the trajectory prediction module is further configured to perform feature extraction on the first trajectory matrix and the first map matrix through an embedded network in the trained trajectory prediction model, obtain a channel number of a last convolutional layer of the embedded network, and obtain a first feature matrix corresponding to the first trajectory matrix and a second feature matrix corresponding to the first map matrix according to the channel number; merging the first characteristic matrix and the second characteristic matrix to obtain a combined matrix; and adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
- The device according to claim 7, wherein the trajectory acquisition module is further configured to acquire drive test data, and perform sensing processing on the drive test data to obtain trajectory information of an obstacle to be predicted in the drive test data; and sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
- A computer device comprising one or more processors and memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:acquiring a motion trail of an obstacle to be predicted;determining target map information corresponding to the obstacle to be predicted according to the motion track;converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; andinputting the first track matrix and the first map matrix into a trained track prediction model, performing embedding processing on the first track matrix and the first map matrix to obtain a target matrix, performing feature extraction on the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: the trained trajectory prediction model comprises a multi-head attention network, the multi-head attention network comprises a one-dimensional convolutional layer, and the one-dimensional convolutional layer is used for respectively extracting the features of the target matrix in the abscissa direction and the features of the target matrix in the ordinate direction.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: respectively extracting features of the first track matrix and the first map matrix through an embedded network in the trained track prediction model, acquiring the number of channels of the last layer of convolution layer of the embedded network, and acquiring a first feature matrix corresponding to the first track matrix and a second feature matrix corresponding to the first map matrix according to the number of channels; merging the first characteristic matrix and the second characteristic matrix to obtain a combined matrix; and adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: acquiring drive test data, and performing sensing processing on the drive test data to obtain track information of an obstacle to be predicted in the drive test data; and sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: determining a corresponding lane central line according to the motion track; and sampling the lane center line to obtain target map information corresponding to the obstacle to be predicted.
- One or more computer storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:acquiring a motion trail of an obstacle to be predicted;determining target map information corresponding to the obstacle to be predicted according to the motion trail;converting the motion track into a corresponding first track matrix, and converting the target map information into a corresponding first map matrix; andinputting the first track matrix and the first map matrix into a trained track prediction model, embedding the first track matrix and the first map matrix to obtain a target matrix, extracting features of the target matrix based on a multi-head attention mechanism to obtain output features, and performing regression processing on the output features to obtain a predicted track of the obstacle to be detected.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: the trained trajectory prediction model comprises a multi-head attention network, the multi-head attention network comprises a one-dimensional convolutional layer, and the one-dimensional convolutional layer is used for respectively extracting the features of the target matrix in the abscissa direction and the features of the target matrix in the ordinate direction.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: respectively extracting features of the first track matrix and the first map matrix through an embedded network in the trained track prediction model, acquiring the number of channels of the last convolution layer of the embedded network, and acquiring a first feature matrix corresponding to the first track matrix and a second feature matrix corresponding to the first map matrix according to the number of channels; merging the first characteristic matrix and the second characteristic matrix to obtain a combined matrix; and adding characteristic parameters into the combined matrix, and performing position embedding processing on the combined matrix after the characteristic parameters are added to obtain a target matrix.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: acquiring drive test data, and sensing the drive test data to obtain track information of an obstacle to be predicted in the drive test data; and sampling the track information of the obstacle to be predicted according to a preset sampling condition to obtain a motion track corresponding to the obstacle to be predicted.
- A vehicle comprising performing a trajectory prediction method according to any one of claims 1-6.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2021/088937 WO2022222095A1 (en) | 2021-04-22 | 2021-04-22 | Trajectory prediction method and apparatus, and computer device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115917559A true CN115917559A (en) | 2023-04-04 |
Family
ID=83723549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202180050157.3A Pending CN115917559A (en) | 2021-04-22 | 2021-04-22 | Trajectory prediction method, apparatus, computer device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115917559A (en) |
| WO (1) | WO2022222095A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116558540A (en) * | 2023-07-11 | 2023-08-08 | 新石器慧通（北京）科技有限公司 | Model training method and device, and track generating method and device |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115047894B (en) * | 2022-04-14 | 2023-09-15 | 中国民用航空总局第二研究所 | Unmanned aerial vehicle track measuring and calculating method, electronic equipment and storage medium |
| CN116071925B (en) * | 2023-02-13 | 2024-04-12 | 北京爱芯科技有限公司 | Track prediction method and device and electronic processing device |
| CN116341721B (en) * | 2023-03-02 | 2023-10-31 | 石家庄铁道大学 | Operation destination prediction method and system based on vehicle track |
| CN116383616A (en) * | 2023-03-31 | 2023-07-04 | 电子科技大学长三角研究院(衢州) | Track GPS coordinate recovery method and frame based on track similarity and deep learning |
| CN116184352B (en) * | 2023-04-26 | 2023-08-22 | 武汉能钠智能装备技术股份有限公司四川省成都市分公司 | Radio frequency target detection system based on track estimation |
| CN116824407A (en) * | 2023-06-21 | 2023-09-29 | 深圳市华赛睿飞智能科技有限公司 | Target detection method, device and equipment based on patrol robot |
| CN117058472B (en) * | 2023-10-12 | 2024-02-20 | 华侨大学 | 3D target detection method, device and equipment based on self-attention mechanism |
| CN117473337B (en) * | 2023-12-25 | 2024-03-22 | 中国科学院自动化研究所 | Track association method, track association device, track association equipment and storage medium |
| CN117493424B (en) * | 2024-01-03 | 2024-03-22 | 湖南工程学院 | Vehicle track prediction method independent of map information |
| CN117874529B (en) * | 2024-03-12 | 2024-05-28 | 浪潮电子信息产业股份有限公司 | Motion trail prediction method, model training method, device, equipment and medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3706034A1 (en) * | 2019-03-06 | 2020-09-09 | Robert Bosch GmbH | Movement prediction of pedestrians useful for autonomous driving |
| CN111238523B (en) * | 2020-04-23 | 2020-08-07 | 北京三快在线科技有限公司 | Method and device for predicting motion trail |
| CN112037506B (en) * | 2020-07-29 | 2021-07-09 | 长安大学 | Vehicle track prediction model construction method, vehicle track prediction method and system |
| CN112015847B (en) * | 2020-10-19 | 2021-03-02 | 北京三快在线科技有限公司 | Obstacle trajectory prediction method and device, storage medium and electronic equipment |
-
2021
- 2021-04-22 CN CN202180050157.3A patent/CN115917559A/en active Pending
- 2021-04-22 WO PCT/CN2021/088937 patent/WO2022222095A1/en active Application Filing
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116558540A (en) * | 2023-07-11 | 2023-08-08 | 新石器慧通（北京）科技有限公司 | Model training method and device, and track generating method and device |
| CN116558540B (en) * | 2023-07-11 | 2023-10-03 | 新石器慧通（北京）科技有限公司 | Model training method and device, and track generating method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022222095A1 (en) | 2022-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115917559A (en) | Trajectory prediction method, apparatus, computer device and storage medium | |
| JP7430277B2 (en) | Obstacle detection method and apparatus, computer device, and computer program | |
| CN112149550B (en) | Automatic driving vehicle 3D target detection method based on multi-sensor fusion | |
| CN111027401B (en) | End-to-end target detection method with integration of camera and laser radar | |
| CN113424079A (en) | Obstacle detection method, obstacle detection device, computer device, and storage medium | |
| CN111160302A (en) | Obstacle information identification method and device based on automatic driving environment | |
| CN113811830B (en) | Trajectory prediction method, apparatus, computer device and storage medium | |
| CN114930401A (en) | Point cloud-based three-dimensional reconstruction method and device and computer equipment | |
| KR102266996B1 (en) | Method and apparatus for limiting object detection area in a mobile system equipped with a rotation sensor or a position sensor with an image sensor | |
| CN113490965A (en) | Image tracking processing method and device, computer equipment and storage medium | |
| CN113592905B (en) | Vehicle driving track prediction method based on monocular camera | |
| CN115187964A (en) | Automatic driving decision-making method based on multi-sensor data fusion and SoC chip | |
| Khurana et al. | Point cloud forecasting as a proxy for 4d occupancy forecasting | |
| JP7468254B2 (en) | Position and orientation calculation method and program | |
| CN111753639A (en) | Perception map generation method and device, computer equipment and storage medium | |
| CN113383283B (en) | Perceptual information processing method, apparatus, computer device, and storage medium | |
| CN117423077A (en) | BEV perception model, construction method, device, equipment, vehicle and storage medium | |
| CN112433193B (en) | Multi-sensor-based mold position positioning method and system | |
| CN115943400B (en) | Track prediction method and device based on time and space learning and computer equipment | |
| CN117706942B (en) | Environment sensing and self-adaptive driving auxiliary electronic control method and system | |
| Yigzaw | An Analysis and Benchmarking in Autoware. AI and OpenPCDet LiDAR-based 3D Object Detection Models | |
| CN117593892B (en) | Method and device for acquiring true value data, storage medium and electronic equipment | |
| CN115994934B (en) | Data time alignment method and device and domain controller | |
| US20230419532A1 (en) | Method, device and computer program for training an artificial neural network for object recognition in images | |
| Xiao et al. | No Longer Getting Lost on Fork Road: Vehicle Off-Route Detection Via Multi-Sensor Integration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |