CN113128381A - Obstacle trajectory prediction method, system and computer storage medium - Google Patents
Obstacle trajectory prediction method, system and computer storage medium Download PDFInfo
- Publication number
- CN113128381A CN113128381A CN202110369182.XA CN202110369182A CN113128381A CN 113128381 A CN113128381 A CN 113128381A CN 202110369182 A CN202110369182 A CN 202110369182A CN 113128381 A CN113128381 A CN 113128381A
- Authority
- CN
- China
- Prior art keywords
- obstacle
- predicted
- time
- neural network
- state information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013528 artificial neural network Methods 0.000 claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 38
- 230000004888 barrier function Effects 0.000 claims abstract description 22
- 230000003068 static effect Effects 0.000 claims abstract description 16
- 239000013598 vector Substances 0.000 claims description 20
- 230000006399 behavior Effects 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 15
- 230000003993 interaction Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 abstract description 6
- 230000007613 environmental effect Effects 0.000 abstract description 6
- 238000004364 calculation method Methods 0.000 abstract description 4
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides a method, a system and a computer storage medium for predicting an obstacle track, wherein the method comprises the following steps: acquiring historical state information from an initial moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle; inputting the vectorized historical state information into a grade map neural network to output a predicted track of the barrier from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the barrier from the observation time to the prediction time and a real track of the barrier from the observation time to the prediction time; and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network. According to the method and the device, effective extraction of environmental characteristics and combined training of the behavior track of the obstacle are realized through fewer parameters and lower calculation consumption, so that an ideal obstacle track prediction effect is achieved.
Description
Technical Field
The invention relates to the technical field of unmanned driving, in particular to a method and a system for predicting an obstacle trajectory and a computer storage medium.
Background
The trajectory prediction of the obstacle means predicting a future trajectory of the obstacle based on a historical movement path of the obstacle. In an autonomous driving scenario, it is desirable to predict the motion trajectory of obstacles around the unmanned vehicle that may affect it, including: motor vehicles, non-motor vehicles, pedestrians, and the like. The specific difficulties faced mainly include the following aspects: 1) the motion trail needs to satisfy traffic rules and physical constraints at the same time; 2) the movement state of the obstacles, the surrounding traffic environment and the like; 3) a plurality of possible reasonable tracks need to be provided for subsequent screening; 4) the traffic road conditions are complex and have various possibilities, and an area without clear lane division range exists, so that the difficulty of predicting the obstacle track is increased.
Disclosure of Invention
In view of this, the present invention provides a method, a system, and a computer storage medium for predicting an obstacle trajectory, which achieve effective extraction of environmental characteristics and joint training of an obstacle behavior trajectory through fewer parameters and lower computation consumption, thereby achieving an ideal obstacle trajectory prediction effect.
In a first aspect, the present invention provides a method for predicting an obstacle trajectory, including:
acquiring historical state information from a starting moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle;
inputting the historical state information after vectorization into a grade map neural network to output a predicted track of the obstacle from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the obstacle from the observation time to the prediction time and a real track of the obstacle from the observation time to the prediction time;
and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network.
The obtaining of the historical state information from the starting time to the observation time includes:
acquiring obstacle state information in a preset range around a vehicle;
obtaining vehicle position information from a starting time to an observation time;
and determining static environment information in a preset range around the vehicle according to the high-precision map and the vehicle position information.
Wherein, the vectorizing the historical state information and inputting the vectorized historical state information into a hierarchical graph neural network comprises:
vectorizing and coding the historical state information;
extracting low-order subgraph features and high-order interaction features according to the vectorized historical state information;
establishing a local feature map according to the low-order subgraph features;
establishing a global feature map according to the local feature map and the high-order interaction features;
inputting the global feature map into the hierarchical map neural network to output a predicted trajectory of the obstacle from the observation time to the predicted time.
Vectorizing and coding the historical state information, wherein the vectorizing and coding comprises the following steps:
extracting target information in the static environment information and the barrier state information, wherein the target information comprises at least one of a departure point coordinate, a termination point coordinate, a target type, a timestamp, a road attribute and a lane speed limit;
and marking the target information as the vector characteristics of each geographical element in the static environment information and each obstacle in the obstacle state information.
Wherein the inputting the global feature map into the hierarchical map neural network to output a predicted trajectory of the obstacle from the observation time to the predicted time comprises:
processing the global feature map through a full connection layer to obtain behavior classification of the barrier and an output vector of a motion track;
and inputting the behavior classification and the output vector of the motion trail into a hierarchical graph neural network so as to output a predicted trail of the obstacle from the observation time to the prediction time.
Wherein the training of the hierarchical graph neural network according to the predicted trajectory of the obstacle from the observation time to the predicted time and the actual trajectory of the obstacle from the observation time to the predicted time further comprises:
calculating a difference value between a predicted track and the real track of the obstacle from the observation time to the predicted time according to a loss function;
and performing multi-task joint training on the grade chart neural network according to the difference value to adjust the grade chart neural network, wherein the multi-task joint training comprises regression training on the movement locus of the obstacle and behavior classification training on the obstacle.
Wherein the loss function is formulated as follows:
L＝Ltraj+αLcls (1)
wherein L istrajIs a loss function of the regression task; l isclsAlpha is a weight adjustment parameter for the loss function of the classification task.
Wherein the multitask joint training of the hierarchical graph neural network according to the difference value to adjust the hierarchical graph neural network comprises:
and performing multi-task joint training on the hierarchical graph neural network according to the difference value to adjust the predicted trajectory of the obstacle from the observation time to the prediction time until the difference value between the predicted trajectory of the obstacle from the observation time to the prediction time and the real trajectory tends to be stable or converged.
In a second aspect, the present invention further provides an obstacle trajectory prediction system, including a memory for storing at least one program instruction and a processor for implementing the obstacle trajectory prediction method as described above by loading and executing the at least one program instruction.
In a third aspect, the present invention also provides a computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement the obstacle trajectory prediction method as described above.
In summary, the method, system and computer storage medium for predicting an obstacle trajectory according to the present invention include: acquiring historical state information from an initial moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle; inputting the vectorized historical state information into a grade map neural network to output a predicted track of the barrier from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the barrier from the observation time to the prediction time and a real track of the barrier from the observation time to the prediction time; and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network. According to the method and the device, effective extraction of environmental characteristics and combined training of the behavior track of the obstacle are realized through fewer parameters and lower calculation consumption, so that an ideal obstacle track prediction effect is achieved.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more clearly understood, the following preferred embodiments are described in detail with reference to the accompanying drawings.
Drawings
Fig. 1 is a schematic flow chart illustrating a method for predicting an obstacle trajectory according to an embodiment of the present invention;
fig. 2 is a control flow chart of an obstacle trajectory prediction method according to an embodiment of the present invention;
fig. 3 is a specific flowchart of an obstacle trajectory prediction method according to an embodiment of the present invention.
Detailed Description
To further explain the technical means and effects of the present invention adopted to achieve the predetermined objects, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.
First embodiment
Fig. 1 is a flowchart illustrating an obstacle trajectory prediction method according to a first embodiment.
As shown in fig. 1, an embodiment of the present invention provides a method for predicting an obstacle trajectory, including:
step 201: acquiring historical state information from an initial moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle;
step 202: inputting the vectorized historical state information into a grade map neural network to output a predicted track of the barrier from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the barrier from the observation time to the prediction time and a real track of the barrier from the observation time to the prediction time;
step 203: and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network.
According to the method, the problem of feature expression and extraction is solved by utilizing vector feature expression in a mode of a Hierarchical Graph Neural Network (HGNN), and the motion trail of the obstacle is finally obtained through the combined training of the classification of the obstacle behaviors and the regression of the motion trail, so that the understanding capability of the automatic driving vehicle to the surrounding environment is improved. It should be noted that, in this embodiment, by acquiring the real operation trajectory of the obstacle within a period of time, sequentially determining the starting time, the observation time, and the predicted time within the period of time, the historical state information from the starting time to the observation time is input to the hierarchical graph neural network, so as to obtain the actual operation trajectory of the obstacle within the period of timeAnd outputting the predicted obstacle track from the observation time to the prediction time. And then comparing the predicted trajectory of the obstacle with the actual trajectory of the obstacle from the observation time to the predicted time. And adjusting the grade map neural network according to the comparison result, outputting the predicted obstacle track again by using the adjusted grade map neural network, continuously comparing with the real track, and training the grade map neural network in such a reciprocating way until an ideal predicted obstacle track is obtained. Assume that the observed historical state input for n obstacles is X ═ X1,X2,...XnThe real future track is Y ═ Y1,Y2,...Yn. The historical position information of the obstacle i is
Wherein t is 1obs，tobsIs the observation time. The future trajectory (true value) is
Where t is tobs+1,...,tpred，tpredIs the predicted time of day. The predicted trajectory is
The unmanned vehicle's understanding of the environment relies on the extraction and expression of environmental features by the unmanned algorithm, which may include high-precision map features as well as surrounding obstacle features. As shown in fig. 2, in step 201, obstacle state information in a preset range around the vehicle is obtained through sensor sensing, then vehicle position information from a start time to an observation time is obtained according to a positioning module, and static environment information in the preset range around the vehicle is determined according to a high-precision map and the vehicle position information.
In step 202, vectorization coding is performed on the historical state information; extracting low-order subgraph features and high-order interaction features according to the vectorized historical state information; establishing a local feature map according to the low-order subgraph features; establishing a global feature map according to the local feature map and the high-order interaction features; and inputting the global feature map into a hierarchical map neural network to output the predicted track of the obstacle from the observation time to the prediction time. Specifically, the original high-precision map road elements, the historical track of the obstacle and other original features are all expressed in a vector form. Then, a sub-graph neural network is established to represent local feature information of each vector, and finally, high-order feature interaction between the sub-graph neural networks is established. The method and the device use the vector to represent the environmental characteristics, including high-precision map road elements, barrier characteristics and the like, and solve the problems of characteristic expression and extraction in a Hierarchical Graph Network (Hierarchical Graph Network) mode. Starting from the original vector features, after the low-level features of the local spatial features are characterized, high-level feature interaction among the low-level features is established, and therefore a hierarchical graph neural network is established.
In one embodiment, vectorizing encoding historical state information includes:
extracting target information in the static environment information and the barrier state information, wherein the target information comprises at least one of a departure point coordinate, a termination point coordinate, a target type, a timestamp, a road attribute and a lane speed limit;
and marking target information as the vector characteristics of each pixel of each region in the static environment information and each obstacle in the obstacle state information.
In one embodiment, inputting a global feature map into a hierarchical map neural network to output a predicted trajectory of an obstacle from an observation time to a predicted time includes:
processing the global feature map through the full connection layer to obtain behavior classification of the barrier and an output vector of a motion track;
and inputting the output vectors of the behavior classification and the motion trail into a hierarchical graph neural network so as to output a predicted trail of the obstacle from the observation time to the prediction time.
In step 202, training a hierarchical graph neural network according to a predicted trajectory of the obstacle from the observation time to the prediction time and a real trajectory of the obstacle from the observation time to the prediction time includes:
calculating a difference value between a predicted track and a real track of the obstacle from the observation time to the predicted time according to the loss function;
and performing multi-task joint training on the grade chart neural network according to the difference value to adjust the grade chart neural network, wherein the multi-task joint training comprises regression training on the motion trail and behavior classification training on the barrier.
The loss function is formulated as follows:
L＝Ltraj+αLcls (1)
wherein L istrajIs a loss function of the regression task; l isclsAlpha is a weight adjustment parameter for the loss function of the classification task.
In one embodiment, the multitask joint training of the hierarchical graph neural network according to the difference value to adjust the hierarchical graph neural network comprises the following steps:
and performing multi-task joint training on the hierarchical graph neural network according to the difference value to adjust the predicted track of the obstacle from the observation time to the prediction time until the difference value between the predicted track and the real track of the obstacle from the observation time to the prediction time tends to be stable or converged.
As shown in fig. 3, the obstacle trajectory prediction flow of the present embodiment is specifically described as follows:
1) the method comprises the steps of obtaining information X of n obstacles in a certain range around the unmanned vehicle through sensors such as a camera, a laser radar and a millimeter wave radar which are arranged on the unmanned vehicle1,X2,...XnWherein
Is historical state information, t 1obs. For trajectory prediction, historical state information is used to predict future motion trajectoriesFor important reference, the obtained historical state information needs to be saved in the feature historical state variable.
2) The positioning module gives the position information of the current unmanned vehicle and stores the historical state information of the unmanned vehicle in a characteristic historical state variable. And according to the positioning result, obtaining static environment information in a certain range around the unmanned vehicle from the high-precision map, such as a lane of an intersection, a pedestrian crossing and the like.
3) Vectorizing the high-precision map features and the obstacle features to be applied to a subsequent HGNN. Firstly, map elements within a preset range around an obstacle, such as a lane, a lane center line, a pedestrian crossing and the like, are acquired. Representing an element i as a vector
Wherein,
and
as starting and ending points of the vector, which may be two-dimensional or three-dimensional coordinate representation, aiFor attribute feature representation, such as object type (e.g., lane center line, crosswalk, obstacle, etc.), timestamp, road attribute feature, lane speed limit, etc., j is used to identify the vector.
4) And inputting the vectorized features into the HGNN to extract low-order sub-graph features and high-order interactive features. HGNN first integrates all vector features viGNN is used for coding the sub-graph nodes to obtain a local feature graph. And then establishing a global feature map by utilizing the interactive relation among the nodes of the high-order GNN coding subgraph.
5) Each node of the Fully-Connected Layer is Connected with all nodes of the previous Layer, and is used for integrating the extracted characteristics to extract and integrate useful information. And (4) obtaining the output vector of barrier behavior classification and motion trail regression after the characteristic expression result of the HGNN passes through a full connection layer.
This exampleIn the middle, the behavior of the obstacle is classified into driving behaviors such as left turn, straight running, right turn, turning around and the like, and the motion trajectory is the motion trajectory of the obstacle in a period of time in the future (the time range can be set according to the actual situation). Using true track from observed to predicted time
Training a grade graph neural network to output a predicted trajectory of the obstacle over a period of time in the future, wherein t is tobs+1,...,tpredTo obtain a true value annotation. The multi-task joint training can use various machine learning classifiers such as softmax, SVM and the like, and the training objective function is L ═ Ltraj+αLcls，LtrajThe calculation formula is a loss function of the regression task and is as follows:
Lclsthe weights of the two are adjusted by using a parameter alpha as a loss function of the classification task. Training, verifying, obtaining test data and training a network model according to requirements to finally obtain a track prediction result of the obstacle
The method trains the behavior and the movement track of the barrier jointly. The regression prediction of the independent motion trail is very sensitive to the interference of input information, and the introduction of classification information of future motion behaviors of the obstacle solves the problem. The future movement behaviors of the obstacle, such as left turning, straight going and right turning, play a crucial role in predicting the trajectory of the obstacle, can improve the accuracy of trajectory prediction, can effectively reduce the interference of input noise, and improves the robustness of the system. Meanwhile, the resource consumption is greatly reduced, the required information is directly extracted from the structured map and expressed by the most basic vector, the number of map neural network model parameters is very small, the resource consumption is lower, and the better effect is achieved.
The obstacle trajectory prediction method provided by the embodiment of the invention comprises the following steps: acquiring historical state information from an initial moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle; inputting the vectorized historical state information into a grade map neural network to output a predicted track of the barrier from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the barrier from the observation time to the prediction time and a real track of the barrier from the observation time to the prediction time; and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network. According to the method and the device, effective extraction of environmental characteristics and combined training of the behavior track of the obstacle are realized through fewer parameters and lower calculation consumption, so that an ideal obstacle track prediction effect is achieved.
An embodiment of the present invention further provides an obstacle trajectory prediction system, which includes a memory and a processor, where the memory is configured to store at least one program instruction, and the processor is configured to implement the obstacle trajectory prediction method described above by loading and executing the at least one program instruction.
The specific process of executing the above method steps in this embodiment is described in detail in the related description of the above embodiment, and is not described again here.
The embodiment of the invention also provides a computer storage medium, wherein the computer storage medium is stored with computer program instructions; the computer program instructions, when executed by the processor, implement the obstacle trajectory prediction method as described above.
The specific process of executing the above method steps in this embodiment is described in detail in the related description of the above embodiment, and is not described again here.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, including not only those elements listed, but also other elements not expressly listed.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. An obstacle trajectory prediction method, comprising:
acquiring historical state information from a starting moment to an observation moment, wherein the historical state information comprises static environment information and obstacle state information around a vehicle;
inputting the historical state information after vectorization into a grade map neural network to output a predicted track of the obstacle from the observation time to the prediction time, and training the grade map neural network according to the predicted track of the obstacle from the observation time to the prediction time and a real track of the obstacle from the observation time to the prediction time;
and outputting the predicted track of the obstacle after the predicted moment through the trained grade graph neural network.
2. The method of predicting an obstacle trajectory according to claim 1, wherein the obtaining of the historical state information from the start time to the observation time includes:
acquiring obstacle state information in a preset range around a vehicle;
obtaining vehicle position information from a starting time to an observation time;
and determining static environment information in a preset range around the vehicle according to the high-precision map and the vehicle position information.
3. The method according to claim 1, wherein the vectorizing the historical state information and inputting the vectorized historical state information into a hierarchical graph neural network comprises:
vectorizing and coding the historical state information;
extracting low-order subgraph features and high-order interaction features according to the vectorized historical state information;
establishing a local feature map according to the low-order subgraph features;
establishing a global feature map according to the local feature map and the high-order interaction features;
inputting the global feature map into the hierarchical map neural network to output a predicted trajectory of the obstacle from the observation time to the predicted time.
4. The method according to claim 3, wherein the vectorized encoding of the historical state information comprises:
extracting target information in the static environment information and the barrier state information, wherein the target information comprises at least one of a departure point coordinate, a termination point coordinate, a target type, a timestamp, a road attribute and a lane speed limit;
and marking the target information as the vector characteristics of each geographical element in the static environment information and each obstacle in the obstacle state information.
5. The method of predicting an obstacle trajectory according to claim 3, wherein said inputting the global feature map into the hierarchical map neural network to output a predicted trajectory of an obstacle from the observation time to the predicted time comprises:
processing the global feature map through a full connection layer to obtain behavior classification of the barrier and an output vector of a motion track;
and inputting the behavior classification and the output vector of the motion trail into a hierarchical graph neural network so as to output a predicted trail of the obstacle from the observation time to the prediction time.
6. The method according to claim 1 or 3, wherein the training of the hierarchical neural network based on the predicted trajectory of the obstacle from the observation time to the predicted time and the actual trajectory of the obstacle from the observation time to the predicted time comprises:
calculating a difference value between a predicted track and the real track of the obstacle from the observation time to the predicted time according to a loss function;
and performing multi-task joint training on the grade map neural network according to the difference value to adjust the grade map neural network, wherein the multi-task joint training comprises regression training on the movement track of the obstacle and behavior classification training on the obstacle.
7. The obstacle trajectory prediction method according to claim 6, wherein the loss function is formulated as follows:
L＝Ltraj+αLcls (1)
wherein L istrajIs a loss function of the regression task; l isclsAlpha is a weight adjustment parameter for the loss function of the classification task.
8. The method of predicting an obstacle trajectory according to claim 6, wherein the multitask joint training the rank map neural network according to the difference values to adjust the rank map neural network comprises:
and performing multi-task joint training on the hierarchical graph neural network according to the difference value to adjust the predicted trajectory of the obstacle from the observation time to the prediction time until the difference value between the predicted trajectory of the obstacle from the observation time to the prediction time and the real trajectory tends to be stable or converged.
9. An obstacle trajectory prediction system comprising a memory for storing at least one program instruction and a processor for implementing the obstacle trajectory prediction method of any one of claims 1 to 8 by loading and executing the at least one program instruction.
10. A computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement the method of obstacle trajectory prediction according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369182.XA CN113128381A (en) | 2021-04-06 | 2021-04-06 | Obstacle trajectory prediction method, system and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369182.XA CN113128381A (en) | 2021-04-06 | 2021-04-06 | Obstacle trajectory prediction method, system and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113128381A true CN113128381A (en) | 2021-07-16 |
Family
ID=76775193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110369182.XA Pending CN113128381A (en) | 2021-04-06 | 2021-04-06 | Obstacle trajectory prediction method, system and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113128381A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113753077A (en) * | 2021-08-17 | 2021-12-07 | 北京百度网讯科技有限公司 | Method and device for predicting movement locus of obstacle and automatic driving vehicle |
| CN114194213A (en) * | 2021-12-29 | 2022-03-18 | 北京三快在线科技有限公司 | Target object trajectory prediction method and device, storage medium and electronic equipment |
| CN114407916A (en) * | 2021-12-16 | 2022-04-29 | 阿波罗智联(北京)科技有限公司 | Vehicle control and model training method and device, vehicle, equipment and storage medium |
| CN115540893A (en) * | 2022-11-30 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN116071925A (en) * | 2023-02-13 | 2023-05-05 | 北京爱芯科技有限公司 | Track prediction method and device and electronic processing device |
| CN116499487A (en) * | 2023-06-28 | 2023-07-28 | 新石器慧通（北京）科技有限公司 | Vehicle path planning method, device, equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111190427A (en) * | 2020-04-10 | 2020-05-22 | 北京三快在线科技有限公司 | Method and device for planning track |
| CN111783262A (en) * | 2020-06-17 | 2020-10-16 | 北京航空航天大学 | Road network representation learning method based on hierarchical graph neural network |
| CN111931905A (en) * | 2020-07-13 | 2020-11-13 | 江苏大学 | Graph convolution neural network model and vehicle track prediction method using same |
| CN112015847A (en) * | 2020-10-19 | 2020-12-01 | 北京三快在线科技有限公司 | Obstacle trajectory prediction method and device, storage medium and electronic equipment |
| US20210009163A1 (en) * | 2019-07-08 | 2021-01-14 | Uatc, Llc | Systems and Methods for Generating Motion Forecast Data for Actors with Respect to an Autonomous Vehicle and Training a Machine Learned Model for the Same |
| CN112364997A (en) * | 2020-12-08 | 2021-02-12 | 北京三快在线科技有限公司 | Method and device for predicting track of obstacle |
| CN112417756A (en) * | 2020-11-13 | 2021-02-26 | 清华大学苏州汽车研究院（吴江） | Interactive simulation test system of automatic driving algorithm |
-
2021
- 2021-04-06 CN CN202110369182.XA patent/CN113128381A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210009163A1 (en) * | 2019-07-08 | 2021-01-14 | Uatc, Llc | Systems and Methods for Generating Motion Forecast Data for Actors with Respect to an Autonomous Vehicle and Training a Machine Learned Model for the Same |
| CN111190427A (en) * | 2020-04-10 | 2020-05-22 | 北京三快在线科技有限公司 | Method and device for planning track |
| CN111783262A (en) * | 2020-06-17 | 2020-10-16 | 北京航空航天大学 | Road network representation learning method based on hierarchical graph neural network |
| CN111931905A (en) * | 2020-07-13 | 2020-11-13 | 江苏大学 | Graph convolution neural network model and vehicle track prediction method using same |
| CN112015847A (en) * | 2020-10-19 | 2020-12-01 | 北京三快在线科技有限公司 | Obstacle trajectory prediction method and device, storage medium and electronic equipment |
| CN112417756A (en) * | 2020-11-13 | 2021-02-26 | 清华大学苏州汽车研究院（吴江） | Interactive simulation test system of automatic driving algorithm |
| CN112364997A (en) * | 2020-12-08 | 2021-02-12 | 北京三快在线科技有限公司 | Method and device for predicting track of obstacle |
Non-Patent Citations (1)
| Title |
|---|
| NING WU等: "Learning Effective Road Network Representation with Hierarchical Graph Neural Networks", 《THE 26TH ACM SIGKDD INTERNATIONAL CONFERENCE》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113753077A (en) * | 2021-08-17 | 2021-12-07 | 北京百度网讯科技有限公司 | Method and device for predicting movement locus of obstacle and automatic driving vehicle |
| EP4140845A1 (en) * | 2021-08-17 | 2023-03-01 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method and apparatus for predicting motion track of obstacle and autonomous vehicle |
| CN114407916A (en) * | 2021-12-16 | 2022-04-29 | 阿波罗智联(北京)科技有限公司 | Vehicle control and model training method and device, vehicle, equipment and storage medium |
| CN114407916B (en) * | 2021-12-16 | 2024-01-23 | 阿波罗智联(北京)科技有限公司 | Vehicle control and model training method and device, vehicle, equipment and storage medium |
| CN114194213A (en) * | 2021-12-29 | 2022-03-18 | 北京三快在线科技有限公司 | Target object trajectory prediction method and device, storage medium and electronic equipment |
| CN115540893A (en) * | 2022-11-30 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN115540893B (en) * | 2022-11-30 | 2023-03-14 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN116071925A (en) * | 2023-02-13 | 2023-05-05 | 北京爱芯科技有限公司 | Track prediction method and device and electronic processing device |
| CN116071925B (en) * | 2023-02-13 | 2024-04-12 | 北京爱芯科技有限公司 | Track prediction method and device and electronic processing device |
| CN116499487A (en) * | 2023-06-28 | 2023-07-28 | 新石器慧通（北京）科技有限公司 | Vehicle path planning method, device, equipment and medium |
| CN116499487B (en) * | 2023-06-28 | 2023-09-05 | 新石器慧通（北京）科技有限公司 | Vehicle path planning method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113128381A (en) | Obstacle trajectory prediction method, system and computer storage medium | |
| JP7532615B2 (en) | Planning for autonomous vehicles | |
| CN111771135B (en) | LIDAR positioning using RNN and LSTM for time smoothing in autonomous vehicles | |
| JP7086111B2 (en) | Feature extraction method based on deep learning used for LIDAR positioning of autonomous vehicles | |
| JP7060625B2 (en) | LIDAR positioning to infer solutions using 3DCNN network in self-driving cars | |
| CN112015847B (en) | Obstacle trajectory prediction method and device, storage medium and electronic equipment | |
| CN112347993B (en) | Expressway vehicle behavior and track prediction method based on vehicle-unmanned aerial vehicle cooperation | |
| EP3959576A1 (en) | Methods and systems for trajectory forecasting with recurrent neural networks using inertial behavioral rollout | |
| Fernando et al. | Going deeper: Autonomous steering with neural memory networks | |
| Masmoudi et al. | A reinforcement learning framework for video frame-based autonomous car-following | |
| US20210397198A1 (en) | Enhanced vehicle operation | |
| Zipfl et al. | Relation-based motion prediction using traffic scene graphs | |
| CN116448134B (en) | Vehicle path planning method and device based on risk field and uncertain analysis | |
| Masmoudi et al. | Autonomous car-following approach based on real-time video frames processing | |
| KR20230024392A (en) | Driving decision making method and device and chip | |
| Yoo et al. | Gin: Graph-based interaction-aware constraint policy optimization for autonomous driving | |
| Kuhnt et al. | Towards a unified traffic situation estimation model—Street-dependent behaviour and motion models | |
| CN113119996B (en) | Trajectory prediction method and apparatus, electronic device and storage medium | |
| Zhou et al. | An autonomous navigation approach for unmanned vehicle in off-road environment with self-supervised traversal cost prediction | |
| Fennessy | Autonomous vehicle end-to-end reinforcement learning model and the effects of image segmentation on model quality | |
| Liu et al. | PnP: Integrated Prediction and Planning for Interactive Lane Change in Dense Traffic | |
| CN114527759B (en) | End-to-end driving method based on hierarchical reinforcement learning | |
| US20230082654A1 (en) | System and method for inferring driving constraints from demonstrations | |
| Singh et al. | Motion Planning of the Autonomous Vehicles with Multi-view Images and GRUs | |
| Nino | Context and Behavioral Analysis for Pedestrians in the Domain of Self-Driving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210716 |