CN113811830A - Trajectory prediction method, apparatus, computer device and storage medium - Google Patents
Trajectory prediction method, apparatus, computer device and storage medium Download PDFInfo
- Publication number
- CN113811830A CN113811830A CN201980037489.0A CN201980037489A CN113811830A CN 113811830 A CN113811830 A CN 113811830A CN 201980037489 A CN201980037489 A CN 201980037489A CN 113811830 A CN113811830 A CN 113811830A
- Authority
- CN
- China
- Prior art keywords
- information
- rendering
- map
- track
- obstacles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
Abstract
A trajectory prediction method, comprising: acquiring historical track information and current position information of a plurality of obstacles in the current environment; rendering the historical track information to obtain a track rendering graph; extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions; splicing the track rendering map and the map element rendering map according to the multiple channel dimensions to obtain a spliced image matrix; inputting the spliced image matrix into a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
Description
The application relates to a trajectory prediction method, a trajectory prediction device, a computer device and a storage medium.
The development of the automatic driving technology is promoted by the development of the artificial intelligence technology. In the automatic driving process, it is very necessary to predict the trajectory of an obstacle in the surrounding environment within a certain time. By predicting the future track of the obstacle, the vehicle can recognize the intention of the obstacle earlier, and plan a driving route and driving speed according to the intention of the obstacle, so that collision is avoided, and safety accidents are reduced. The future trajectory of an obstacle is influenced by a number of factors. In a traditional track prediction mode, the track of an obstacle is predicted according to self information of the obstacle, and the self information of the obstacle is only a part of influence factors, so that the accuracy of the predicted obstacle track is low.
Disclosure of Invention
According to various embodiments disclosed herein, a trajectory prediction method, apparatus, computer device, and storage medium capable of improving accuracy of trajectory prediction are provided.
A trajectory prediction method, comprising:
acquiring historical track information and current position information of a plurality of obstacles in the current environment;
rendering the historical track information to obtain a track rendering graph;
extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;
splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;
inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and
and predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
A trajectory prediction device comprising:
the acquisition module is used for acquiring historical track information and current position information of a plurality of obstacles in the current environment;
the first rendering module is used for rendering the historical track information to obtain a track rendering graph;
the second rendering module is used for extracting the map elements from the current position information and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;
the splicing module is used for splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;
the extraction module is used for inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and
and the prediction module is used for predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the processors, cause the one or more processors to perform the steps of:
acquiring historical track information and current position information of a plurality of obstacles in the current environment;
rendering the historical track information to obtain a track rendering graph;
extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;
splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;
inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and
and predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
acquiring historical track information and current position information of a plurality of obstacles in the current environment;
rendering the historical track information to obtain a track rendering graph;
extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;
splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;
inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and
and predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description and drawings, and from the claims.
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a diagram of an application environment of a trajectory prediction method in one or more embodiments.
FIG. 2 is a flow diagram of a trajectory prediction method in one or more embodiments.
FIG. 3 is a schematic diagram of a stitched image matrix in one or more embodiments.
FIG. 4 is a flowchart illustrating steps of rendering historical track information to obtain a track rendering map in one or more embodiments.
FIG. 5 is a flowchart illustrating steps in one or more embodiments for rendering map elements into corresponding map element renderings according to a plurality of channel dimensions.
FIG. 6 is a block diagram of a trajectory prediction device in one or more embodiments.
FIG. 7 is a block diagram of a computer device in one or more embodiments.
In order to make the technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The trajectory prediction method provided by the application can be applied to the application environment shown in fig. 1. The vehicle-mounted sensor 102 transmits the acquired information to be detected to the computer device 104. The vehicle-mounted sensor may be a laser radar or a vehicle-mounted camera. The computer device 104 processes the information to be detected to obtain historical trajectory information of a plurality of obstacles in the current environment. The on-board computer device may be referred to simply as a computer device. The in-vehicle locator 106 transmits the collected current location information to the computer device 104. And rendering the historical track information by the computer equipment 104 to obtain a track rendering graph. The computer device 104 extracts the map elements in the current location information and renders the map elements into corresponding map element renderings according to the plurality of channel dimensions. The computer device 104 splices the trajectory rendering map and the map element rendering map according to the multiple channel dimensions to obtain a spliced image matrix. The computer device 104 inputs the stitched image matrix into the trained feature extractor, and performs feature extraction on the stitched image matrix through the feature extractor to obtain a feature extraction result. The computer device 104 predicts trajectories of the plurality of obstacles within a preset time period from the feature extraction result.
In one embodiment, as shown in fig. 2, a trajectory prediction method is provided, which is exemplified by the method applied to the computer device in fig. 1, and includes the following steps:
In the process of automatic driving of the vehicle, the vehicle-mounted sensor can send collected information to be detected to the computer equipment, and the computer equipment processes the information to be detected to obtain historical track information of a plurality of obstacles. The vehicle-mounted sensor can be used for collecting information to be detected, and historical track information of a plurality of obstacles can be obtained through the detector and the tracker. The vehicle-mounted tracker transmits historical trajectory information of a plurality of obstacles to the computer device. And the vehicle-mounted positioner transmits the acquired current position information to the computer equipment. For example, the vehicle-mounted locator may be a GPS (Global Positioning System) locator. The vehicle-mounted locator can analyze the GPS signal by receiving the GPS signal of the satellite, calculate to obtain corresponding geographical location information, and then transmit the geographical location information to the computer device through a wireless network such as GSM (Global System of Mobile communication)/CDMA (Code Division Multiple Access).
And step 204, rendering the historical track information to obtain a track rendering graph.
And rendering the acquired historical track information of the plurality of obstacles to a characteristic diagram by the computer equipment to obtain a track rendering diagram. The historical trajectory information may be historical trajectories per frame for a plurality of obstacles. And rendering the historical track information of the plurality of obstacles in the current frame by the computer equipment to obtain a track rendering map. The color of the obstacle in each frame in the track rendering map changes along with the distance from the current frame, and the color of the obstacle is lighter as the distance from the current frame is longer.
And step 206, extracting the map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to the multiple channel dimensions.
And step 208, splicing the track rendering map and the map element rendering map according to the multiple channel dimensions to obtain a spliced image matrix.
And the computer equipment acquires the current position information acquired by the vehicle-mounted locator. The current position information may be position information of the vehicle in the high-precision map at the current time. The current location information may be expressed in terms of latitude and longitude. The computer device extracts the map element in the current location information. The map elements may include lane lines, center lines, sidewalks, stop lines, etc. information. The computer equipment can render the extracted map elements according to the multiple channel dimensions, and render the map elements into map element rendering graphs corresponding to the channel dimensions. When the local graphic elements are different, the channel dimensions corresponding to the map elements may also be different. The channel dimensions may include color channels, element channels, and the like. The color channels may include three channels, red, green, and blue. The element lanes may include lane line lanes, center line lanes, and sidewalk lanes, among others.
After obtaining the trajectory rendering map and the map element rendering map, the computer device can splice the trajectory rendering map and the map element rendering map. And the computer equipment determines the corresponding channel dimension of the track rendering map and the map element rendering map, and carries out image splicing on the track rendering map and the map element rendering map on the corresponding channel dimension so as to obtain a spliced image matrix. The stitched image matrix may be a complete image including the trajectory rendering map and the map element rendering map. A schematic diagram of the stitched image matrix may be as shown in fig. 3. The white circles in the figure may represent a vehicle. The line formed by the plurality of white circles indicates the trajectory of the vehicle. The lines represent lane lines. The intersection of the lines represents the centerline.
In one embodiment, the computer device may also pre-process the trajectory rendering graph and the map element rendering graph before stitching the trajectory rendering graph and the map element rendering graph. Specifically, the computer device may perform filtering processing on the trajectory rendering map and the map element rendering map to obtain the trajectory rendering map and the map element rendering map after the filtering processing. The computer equipment can obtain smooth track rendering maps and map element rendering maps by filtering the track rendering maps and the map element rendering maps, can remove noise, and is favorable for improving the accuracy of subsequent feature extraction.
And step 210, inputting the spliced image matrix into a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result.
And 212, performing regression prediction on the feature extraction result to obtain the tracks of the plurality of obstacles in a preset time period.
The computer device has been pre-trained with a feature extractor prior to obtaining historical trajectory information and current location information for a plurality of obstacles in a current environment. The feature extractor is obtained by training the convolutional neural network model according to the sample data. The feature extractor may include a plurality of network layers. For example, an input layer, a convolutional layer, a pooling layer, and a fully-connected layer may be included.
And the computer equipment splices the track rendering map and the map element rendering map according to the multiple channel dimensions to obtain a spliced image matrix, and then can call the trained feature extractor to input the spliced image matrix into the trained feature extractor. And the computer equipment extracts the image characteristic information and the context characteristic information corresponding to the spliced image matrix through the characteristic extractor, and then outputs a characteristic extraction result corresponding to the spliced image matrix through a full connection layer of the characteristic extractor.
The computer equipment can calculate the feature extraction result in a regression prediction mode to obtain the track of the plurality of obstacles in the preset time period. The regression prediction may be to predict the position coordinates of the obstacle within a preset time period from a correlation or causal relationship between the feature extraction results. The position coordinates of the obstacle at any one time within the preset time period may be represented by P (x, y). For example, the preset time period may be 5 s.
In the embodiment, after acquiring historical track information and current position information of a plurality of obstacles in the current environment, the computer device renders the historical track information of the plurality of obstacles into the track rendering map, so that the information of the obstacles and the surrounding environment can be obtained, the influence factors of the track can be considered from multiple aspects, and the accuracy of track prediction can be improved. The computer device renders the map elements in the current location information into a map element rendering map according to the plurality of channel dimensions. The current position of the barrier can be visually and accurately rendered through the channel dimension corresponding to the map element, and the subsequent trajectory prediction is facilitated. And the computer equipment splices the track rendering map and the map element rendering map according to the multiple channel dimensions, and inputs the spliced image matrix into the trained feature extractor for feature extraction to obtain a feature extraction result. The method realizes the combination of various influence factors of the obstacle track, and further improves the comprehensiveness of the feature extraction result. The computer equipment carries out regression prediction on the feature extraction result, and the obtained feature extraction result contains the track of the historical frames of the plurality of obstacles, so that the range of the environmental information is expanded, the track prediction is realized according to influence factors in various aspects, and the accuracy of the track prediction is improved.
In one embodiment, as shown in fig. 4, the step of rendering the historical track information to obtain a track rendering map includes:
And step 404, fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to the plurality of obstacles.
The computer device obtains historical trajectory information for a plurality of obstacles. The historical trajectory information may include a trajectory per frame that is a history of each obstacle. The computer equipment acquires time corresponding to each historical frame track according to each historical frame track of each obstacle, and obtains historical time sequence information of a plurality of obstacles according to the time corresponding to each historical frame track of each obstacle. The historical time sequence information may include a track corresponding to each obstacle in each historical frame, which is generated according to the time sequence. And the computer equipment determines a rendering channel corresponding to the current frame according to the historical track information, and fuses historical time sequence information corresponding to the multiple obstacles in the current frame into one image according to the corresponding rendering channel, so that a track rendering graph corresponding to the multiple obstacles is obtained.
In this embodiment, the computer determines historical timing information according to the historical trajectory information, and fuses the historical timing information in the current frame to obtain trajectory rendering maps corresponding to the multiple obstacles. The historical time sequence information can be fused into one image, and the overall analysis of the track of the obstacle is facilitated. Meanwhile, historical track information of the obstacles does not need to be rendered into independent images one by one, and computing resources of computer equipment are effectively saved.
In one embodiment, as shown in fig. 5, rendering map elements into corresponding map element renderings according to a plurality of channel dimensions includes:
The computer device looks up the map element from the current location information. The current position information may be acquired in a high-precision map. The map elements may include lane lines, center lines, sidewalks, stop lines, etc. information. The channel dimensions may include color channels, element channels, and the like. The color channels may include three channels, red, green, and blue. The element lanes may include lane line lanes, center line lanes, and sidewalk lanes, among others. And the computer equipment identifies whether the map element corresponding to the channel dimension exists in the acquired map elements according to each channel dimension. When the map elements corresponding to the channel dimension exist, rendering the map elements to the map element rendering graphs corresponding to the channel dimension, and after all the map elements are rendered, obtaining the map information rendering graphs around the obstacles according to the map element rendering graphs. For example, the computer device identifies whether a center line exists in the map element according to the center line channel, and when the center line exists, the center line in the map element is rendered into a center line rendering graph corresponding to the center line channel.
In this embodiment, the computer device identifies a corresponding map element according to each channel dimension, renders the map element into a map element rendering map corresponding to the channel dimension, and can visually display map information around the obstacle. Meanwhile, the map element rendering graph and the track rendering graph can be spliced subsequently according to the channel dimension.
In one embodiment, the feature extractor includes a plurality of network layers, and performs feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result, including: extracting an image vector and a context vector in the spliced image matrix through an input layer of a feature extractor; inputting the image vector and the context vector into the convolution layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector; inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; inputting the image characteristic information subjected to the dimensionality reduction processing into a full-connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
And after the computer equipment obtains the spliced image matrix, calling a feature extractor, and inputting the spliced image matrix into the feature extractor for feature extraction. The feature extractor is obtained by training the convolutional neural network model according to the sample data. The feature extractor may include a plurality of network layers. For example, an input layer, a convolutional layer, a pooling layer, and a fully-connected layer may be included.
And the computer equipment extracts the image vector and the context vector in the spliced image matrix through an input layer of the feature extractor. The input layer of the feature extractor takes the extracted image vector and context vector as the input of the convolution layer, and extracts corresponding feature information through the convolution layer to obtain image feature information and context feature information. The image feature information may include spatial feature information and temporal feature information. The spatial feature information may include historical speed change information of the obstacle. The time-series characteristic information may include position information and direction information of the obstacle within a preset time period. The convolution layer of the feature extractor takes the image feature information and the context feature information as the input of the pooling layer, and the dimension reduction processing is carried out on the image feature information and the context feature information through the pooling layer. And the pooling layer of the feature extractor takes the image feature information and the context feature information after the dimension reduction processing as the input of the full connection layer, and outputs the feature extraction result corresponding to the spliced image matrix through the full connection layer.
In this embodiment, the computer device extracts an image vector and a context vector in the stitched image matrix through the input layer of the feature extractor, and extracts image feature information corresponding to the image vector and context feature information corresponding to the context vector through the convolution layer, so that interference information in the stitched image matrix can be filtered, and the stitched image matrix is focused to obtain feature information. The computer equipment performs dimension reduction processing on the image characteristic information and the context characteristic information through a pooling layer of the characteristic extractor, can extract main image characteristic information and context characteristics, and avoids the influence of redundant characteristics. And the computer equipment further outputs the feature extraction result corresponding to the image matrix through the full connection layer, so that the accuracy of feature extraction is improved.
In one embodiment, performing regression prediction on the feature extraction result to obtain trajectories of a plurality of obstacles in a preset time period includes: calculating the number of the predicted points according to a preset time period and a preset sampling rate; regression predicting position change information of the plurality of obstacles in a preset time period according to the number of the predicted points and the feature extraction result; and obtaining the tracks of the plurality of obstacles in a preset time period according to the position change information.
And after the computer equipment obtains the feature extraction result, calculating the number of the predicted points according to a preset time period and a preset sampling rate. The number of predicted points may be obtained by comparing a preset time period with a preset sampling rate. For example, if the preset time period is 5s and the preset sampling rate is 0.2, then 25 is obtained from 5/0.2, i.e. the number of predicted points is 25. And the computer equipment performs regression prediction on the position information of each predicted point according to the characteristic extraction result. The position information may be position coordinates of the obstacle. And the computer equipment calculates the position change information of the obstacle in a preset time period according to the position information of the plurality of predicted points. The position change information may be a position offset amount. And the computer equipment further obtains the track of the obstacle in the preset time period according to the position change information.
In this embodiment, the computer device calculates the number of predicted points according to a preset time period and a preset sampling rate, regresses and predicts position change information of the plurality of obstacles in the preset time period according to the number of predicted points and the feature extraction result, and obtains tracks of the plurality of obstacles in the preset time period according to the position change information. Since the feature extraction result includes feature information of a plurality of obstacles, context feature information in a wider range is provided. Meanwhile, the tracks of the plurality of obstacles in the preset time period can be obtained only through one-time prediction, so that the calculated amount is effectively reduced, the track prediction efficiency is improved, and the real-time track prediction of the obstacles is realized.
In one embodiment, obtaining historical trajectory information of obstacles in a current environment comprises: acquiring information to be detected; detecting the information to be detected according to the type of the information to be detected, and determining the barrier in the current environment; and tracking the movement process of the obstacle to acquire the historical track information of the obstacle.
In the automatic driving process, the vehicle-mounted sensor collects information to be detected and sends the collected information to be detected to the computer equipment. And the computer equipment detects the information to be detected according to the type of the information to be detected and determines the barrier information in the environment at the current moment. The vehicle-mounted sensor can be a laser radar or a vehicle-mounted camera. And when the vehicle-mounted sensor is a laser radar, the type of the information to be detected is point cloud data. The computer device may determine the obstacle in the environment at the current time by classifying the point cloud data. When the vehicle-mounted sensor is a vehicle-mounted camera, the type of the information to be detected is an image. The computer device can segment and semantically label the image according to semantic categories and determine obstacles in the current environment. The computer equipment tracks the movement process of the obstacle, predicts the position information of the current moment according to the position information of the obstacle at the previous moment, and compares the predicted position information of the current moment with the actual position information to obtain error information. And correcting the position information at the next moment according to the error information to obtain historical track information of a plurality of obstacles. The computer equipment can acquire various types of information to be detected, determines a corresponding detection mode according to the type of the information to be detected, detects the obstacle in the current environment, tracks the movement process of the obstacle, and acquires the historical track information of the obstacle. Obstacle detection can be flexibly performed, and corresponding historical track information can be acquired.
In one embodiment, as shown in fig. 6, there is provided a trajectory prediction apparatus including: an acquisition module 602, a first rendering module 604, a second rendering module 606, a stitching module 608, an extraction module 610, and a prediction module 612, wherein:
an obtaining module 602, configured to obtain historical trajectory information and current position information of multiple obstacles in a current environment.
The first rendering module 604 is configured to render the historical track information to obtain a track rendering map.
The second rendering module 606 is configured to extract a map element from the current location information, and render the map element into a corresponding map element rendering map according to the multiple channel dimensions.
The stitching module 608 is configured to stitch the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a stitched image matrix.
And the extraction module 610 is configured to input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result.
And the predicting module 612 is configured to predict trajectories of the multiple obstacles within a preset time period according to the feature extraction result.
In one embodiment, the first rendering module 604 is further configured to determine historical timing information according to the historical track information; and fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to a plurality of obstacles.
In one embodiment, the second rendering module 606 is further configured to identify, according to each channel dimension, whether a map element corresponding to the channel dimension exists in the map elements; and when the map elements corresponding to the channel dimensions exist in the map elements, rendering the map elements into the map element rendering graph corresponding to the channel dimensions.
In one embodiment, the extracting module 610 is further configured to extract an image vector and a context vector in the stitched image matrix through an input layer of the feature extractor; inputting the image vector and the context vector into the convolution layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector; inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; and inputting the image characteristic information and the context characteristic information after the dimension reduction processing into a full connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
In one embodiment, the prediction module 612 is further configured to calculate the number of predicted points according to a preset time period and a preset sampling rate; regression predicting position change information of the plurality of obstacles in a preset time period according to the number of the predicted points and the feature extraction result; and obtaining the tracks of the plurality of obstacles in a preset time period according to the position change information.
In one embodiment, the obtaining module 602 is further configured to obtain information to be detected; detecting the information to be detected according to the type of the information to be detected, and determining the barrier in the current environment; and tracking the movement process of the obstacle to acquire the historical track information of the obstacle.
For the specific definition of the trajectory prediction device, reference may be made to the above definition of the trajectory prediction method, which is not described herein again. The modules in the trajectory prediction device can be implemented in whole or in part by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 7. The computer device includes a processor, a memory, a communication interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operating system and execution of computer-readable instructions in the non-volatile storage medium. The database of the computer device is used for storing historical track information and current position information of obstacles. The communication interface of the computer equipment is used for connecting and communicating with the vehicle-mounted sensor and the vehicle-mounted locator. The computer readable instructions, when executed by a processor, implement a trajectory prediction method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of the various method embodiments described above.
One or more non-transitory computer-readable storage media storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the various method embodiments described above.
It will be understood by those of ordinary skill in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a non-volatile computer readable storage medium, and when executed, can include processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (20)
- A trajectory prediction method, comprising:acquiring historical track information and current position information of a plurality of obstacles in the current environment;rendering the historical track information to obtain a track rendering graph;extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; andand predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
- The method of claim 1, wherein the rendering the historical track information to obtain a track rendering map comprises:determining historical time sequence information according to the historical track information; andand fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to a plurality of obstacles.
- The method of claim 1, wherein the rendering the map elements into corresponding map element renderings according to a plurality of channel dimensions comprises:identifying whether a map element corresponding to each channel dimension exists in the map elements according to each channel dimension; andand when the map element corresponding to the channel dimension exists in the map elements, rendering the map elements to a map element rendering graph corresponding to the channel dimension.
- The method according to claim 1, wherein the feature extractor comprises a plurality of network layers, and the extracting features of the stitched image matrix by the feature extractor to obtain a feature extraction result comprises:extracting an image vector and a context vector in the spliced image matrix through an input layer of the feature extractor;inputting the image vector and the context vector into a convolutional layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector;inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; andand inputting the image characteristic information and the context characteristic information after the dimension reduction processing into a full connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
- The method of claim 1, wherein predicting trajectories of a plurality of obstacles within a preset time period according to the feature extraction result comprises:calculating the number of the predicted points according to a preset time period and a preset sampling rate;regression predicting position change information of a plurality of obstacles in a preset time period according to the number of the predicted points and the feature extraction result; andand obtaining the tracks of the plurality of obstacles in a preset time period according to the position change information.
- The method according to any one of claims 1 to 5, wherein the obtaining of historical trajectory information of obstacles in the current environment comprises:acquiring information to be detected;detecting the information to be detected according to the type of the information to be detected, and determining an obstacle in the current environment; andand tracking the movement process of the obstacle to acquire historical track information of the obstacle.
- A trajectory prediction device comprising:the acquisition module is used for acquiring historical track information and current position information of a plurality of obstacles in the current environment;the first rendering module is used for rendering the historical track information to obtain a track rendering graph;the second rendering module is used for extracting the map elements from the current position information and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;the splicing module is used for splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;the extraction module is used for inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; andand the prediction module is used for predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
- The apparatus of claim 7, wherein the first rendering module is further configured to determine historical timing information from the historical track information; and fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to a plurality of obstacles.
- The apparatus of claim 7, wherein the second rendering module is further configured to identify, according to each channel dimension, whether a map element corresponding to the channel dimension exists in the map elements; and when the map element corresponding to the channel dimension exists in the map elements, rendering the map elements to a map element rendering graph corresponding to the channel dimension.
- The apparatus of claim 7, wherein the extraction module is further configured to extract an image vector and a context vector in the stitched image matrix through an input layer of the feature extractor; inputting the image vector and the context vector into a convolutional layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector; inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; and inputting the image characteristic information and the context characteristic information after the dimension reduction processing into a full connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
- A computer device comprising one or more processors and memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:acquiring historical track information and current position information of a plurality of obstacles in the current environment;rendering the historical track information to obtain a track rendering graph;extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; andand predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: determining historical time sequence information according to the historical track information; and fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to a plurality of obstacles.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: identifying whether a map element corresponding to each channel dimension exists in the map elements according to each channel dimension; and when the map element corresponding to the channel dimension exists in the map elements, rendering the map elements to a map element rendering graph corresponding to the channel dimension.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: extracting an image vector and a context vector in the spliced image matrix through an input layer of the feature extractor; inputting the image vector and the context vector into a convolutional layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector; inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; and inputting the image characteristic information and the context characteristic information after the dimension reduction processing into a full connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
- The computer device of claim 11, wherein the processor, when executing the computer readable instructions, further performs the steps of: calculating the number of the predicted points according to a preset time period and a preset sampling rate; regression predicting position change information of a plurality of obstacles in a preset time period according to the number of the predicted points and the feature extraction result; and obtaining the track of the plurality of obstacles in a preset time period according to the position change information.
- One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:acquiring historical track information and current position information of a plurality of obstacles in the current environment;rendering the historical track information to obtain a track rendering graph;extracting map elements from the current position information, and rendering the map elements into corresponding map element rendering graphs according to a plurality of channel dimensions;splicing the track rendering map and the map element rendering map according to a plurality of channel dimensions to obtain a spliced image matrix;inputting the spliced image matrix to a trained feature extractor, and performing feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; andand predicting the tracks of the plurality of obstacles in a preset time period according to the feature extraction result.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: determining historical time sequence information according to the historical track information; and fusing the historical time sequence information in the current frame to obtain a track rendering map corresponding to a plurality of obstacles.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: identifying whether a map element corresponding to each channel dimension exists in the map elements according to each channel dimension; and when the map element corresponding to the channel dimension exists in the map elements, rendering the map elements to a map element rendering graph corresponding to the channel dimension.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: extracting an image vector and a context vector in the spliced image matrix through an input layer of the feature extractor; inputting the image vector and the context vector into a convolutional layer, and extracting image characteristic information corresponding to the image vector and context characteristic information corresponding to the context vector; inputting the image characteristic information and the context characteristic information into a pooling layer, and performing dimension reduction processing on the image characteristic information and the context characteristic information; and inputting the image characteristic information and the context characteristic information after the dimension reduction processing into a full connection layer, and outputting a characteristic extraction result corresponding to the spliced image matrix.
- The storage medium of claim 16, wherein the computer readable instructions, when executed by the processor, further perform the steps of: calculating the number of the predicted points according to a preset time period and a preset sampling rate; regression predicting position change information of a plurality of obstacles in a preset time period according to the number of the predicted points and the feature extraction result; and obtaining the track of the plurality of obstacles in a preset time period according to the position change information.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2019/130188 WO2021134354A1 (en) | 2019-12-30 | 2019-12-30 | Path prediction method and apparatus, computer device, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113811830A true CN113811830A (en) | 2021-12-17 |
| CN113811830B CN113811830B (en) | 2022-05-10 |
Family
ID=76685830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980037489.0A Active CN113811830B (en) | 2019-12-30 | 2019-12-30 | Trajectory prediction method, apparatus, computer device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113811830B (en) |
| WO (1) | WO2021134354A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610020A (en) * | 2022-01-28 | 2022-06-10 | 广州文远知行科技有限公司 | Method, device and equipment for predicting movement locus of obstacle and storage medium |
| CN114648551A (en) * | 2022-05-19 | 2022-06-21 | 武汉深图智航科技有限公司 | Trajectory prediction method and apparatus |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114245177B (en) * | 2021-12-17 | 2024-01-23 | 智道网联科技(北京)有限公司 | Smooth display method and device of high-precision map, electronic equipment and storage medium |
| CN114089775B (en) * | 2022-01-20 | 2022-06-10 | 杭州蓝芯科技有限公司 | Mobile robot obstacle stopping control method and device |
| CN115257727B (en) * | 2022-09-27 | 2022-12-23 | 禾多科技（北京）有限公司 | Obstacle information fusion method and device, electronic equipment and computer readable medium |
| CN115657674B (en) * | 2022-10-26 | 2023-05-05 | 宝开(上海)智能物流科技有限公司 | Distributed path planning method and device based on graph neural network |
| CN115953746B (en) * | 2023-03-13 | 2023-06-02 | 中国铁塔股份有限公司 | Ship monitoring method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2107528A1 (en) * | 2008-04-01 | 2009-10-07 | Telefonaktiebolaget L M Ericsson (PUBL) | Method of and arrangement for rendering a path |
| CN108803617A (en) * | 2018-07-10 | 2018-11-13 | 深圳大学 | Trajectory predictions method and device |
| CN109101690A (en) * | 2018-07-11 | 2018-12-28 | 深圳地平线机器人科技有限公司 | Method and apparatus for rendering the scene in Vehicular automatic driving simulator |
| EP3428857A1 (en) * | 2017-07-14 | 2019-01-16 | Rosemount Aerospace Inc. | Render-based trajectory planning |
| CN109739926A (en) * | 2019-01-09 | 2019-05-10 | 南京航空航天大学 | A kind of mobile object destination prediction technique based on convolutional neural networks |
| CN109885066A (en) * | 2019-03-26 | 2019-06-14 | 北京经纬恒润科技有限公司 | A kind of motion profile prediction technique and device |
| CN109927719A (en) * | 2017-12-15 | 2019-06-25 | 百度在线网络技术（北京）有限公司 | A kind of auxiliary driving method and system based on barrier trajectory predictions |
| CN110047124A (en) * | 2019-04-23 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of render video |
| CN110223318A (en) * | 2019-04-28 | 2019-09-10 | 驭势科技（北京）有限公司 | A kind of prediction technique of multi-target track, device, mobile unit and storage medium |
| CN110428500A (en) * | 2019-07-29 | 2019-11-08 | 腾讯科技（深圳）有限公司 | Track data processing method, device, storage medium and equipment |
| CN112651990A (en) * | 2020-12-25 | 2021-04-13 | 际络科技(上海)有限公司 | Motion trajectory prediction method and system, electronic device and readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8706417B2 (en) * | 2012-07-30 | 2014-04-22 | GM Global Technology Operations LLC | Anchor lane selection method using navigation input in road change scenarios |
| CN106778548B (en) * | 2016-11-30 | 2021-04-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting obstacles |
| CN109901194A (en) * | 2019-03-18 | 2019-06-18 | 爱驰汽车有限公司 | Onboard system, method, equipment and the storage medium of anticollision |
| CN110570664B (en) * | 2019-09-23 | 2023-04-07 | 山东科技大学 | Automatic detection system for highway traffic incident |
-
2019
- 2019-12-30 CN CN201980037489.0A patent/CN113811830B/en active Active
- 2019-12-30 WO PCT/CN2019/130188 patent/WO2021134354A1/en active Application Filing
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2107528A1 (en) * | 2008-04-01 | 2009-10-07 | Telefonaktiebolaget L M Ericsson (PUBL) | Method of and arrangement for rendering a path |
| EP3428857A1 (en) * | 2017-07-14 | 2019-01-16 | Rosemount Aerospace Inc. | Render-based trajectory planning |
| CN109927719A (en) * | 2017-12-15 | 2019-06-25 | 百度在线网络技术（北京）有限公司 | A kind of auxiliary driving method and system based on barrier trajectory predictions |
| CN108803617A (en) * | 2018-07-10 | 2018-11-13 | 深圳大学 | Trajectory predictions method and device |
| CN109101690A (en) * | 2018-07-11 | 2018-12-28 | 深圳地平线机器人科技有限公司 | Method and apparatus for rendering the scene in Vehicular automatic driving simulator |
| CN109739926A (en) * | 2019-01-09 | 2019-05-10 | 南京航空航天大学 | A kind of mobile object destination prediction technique based on convolutional neural networks |
| CN109885066A (en) * | 2019-03-26 | 2019-06-14 | 北京经纬恒润科技有限公司 | A kind of motion profile prediction technique and device |
| CN110047124A (en) * | 2019-04-23 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of render video |
| CN110223318A (en) * | 2019-04-28 | 2019-09-10 | 驭势科技（北京）有限公司 | A kind of prediction technique of multi-target track, device, mobile unit and storage medium |
| CN110428500A (en) * | 2019-07-29 | 2019-11-08 | 腾讯科技（深圳）有限公司 | Track data processing method, device, storage medium and equipment |
| CN112651990A (en) * | 2020-12-25 | 2021-04-13 | 际络科技(上海)有限公司 | Motion trajectory prediction method and system, electronic device and readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610020A (en) * | 2022-01-28 | 2022-06-10 | 广州文远知行科技有限公司 | Method, device and equipment for predicting movement locus of obstacle and storage medium |
| CN114648551A (en) * | 2022-05-19 | 2022-06-21 | 武汉深图智航科技有限公司 | Trajectory prediction method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021134354A1 (en) | 2021-07-08 |
| CN113811830B (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113811830B (en) | Trajectory prediction method, apparatus, computer device and storage medium | |
| CN109934164B (en) | Data processing method and device based on track safety degree | |
| CN113678136A (en) | Obstacle detection method and device based on unmanned technology and computer equipment | |
| WO2022222095A1 (en) | Trajectory prediction method and apparatus, and computer device and storage medium | |
| CN113223317B (en) | Method, device and equipment for updating map | |
| CN111256687A (en) | Map data processing method and device, acquisition equipment and storage medium | |
| CN111507373A (en) | Method and apparatus for performing seamless parameter change | |
| CN111753639A (en) | Perception map generation method and device, computer equipment and storage medium | |
| US20230278587A1 (en) | Method and apparatus for detecting drivable area, mobile device and storage medium | |
| CN114419552A (en) | Illegal vehicle tracking method and system based on target detection | |
| CN113895456A (en) | Intersection driving method and device for automatic driving vehicle, vehicle and medium | |
| CN112445204A (en) | Object movement navigation method and device in construction site and computer equipment | |
| US11531349B2 (en) | Corner case detection and collection for a path planning system | |
| CN114968187A (en) | Platform for perception system development of an autopilot system | |
| CN113160272B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111380536A (en) | Vehicle positioning method and device, electronic equipment and computer readable storage medium | |
| CN113383283A (en) | Perception information processing method and device, computer equipment and storage medium | |
| CN112689234A (en) | Indoor vehicle positioning method and device, computer equipment and storage medium | |
| CN109344776B (en) | Data processing method | |
| CN116907523A (en) | Path planning method, path planning device, computer equipment and storage medium | |
| CN115817466A (en) | Collision risk assessment method and device | |
| CN115257771A (en) | Intersection identification method, electronic device and storage medium | |
| CN110930688A (en) | Planning method and device for vehicle driving path, computer equipment and storage medium | |
| US11928406B2 (en) | Systems and methods for creating infrastructure models | |
| CN110874549A (en) | Target visual field determining method, system, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |