WO2021134354A1

WO2021134354A1 - Path prediction method and apparatus, computer device, and storage medium

Info

Publication number: WO2021134354A1
Application number: PCT/CN2019/130188
Authority: WO
Inventors: 邹晓艺; 何明; 叶茂盛; 吴伟; 许双杰; 许家妙; 曹通易
Original assignee: 深圳元戎启行科技有限公司
Priority date: 2019-12-30
Filing date: 2019-12-30
Publication date: 2021-07-08
Also published as: CN113811830B; CN113811830A

Abstract

A path prediction method, comprising: acquiring historical path information of multiple obstacles in the current environment, and current location information; rendering the historical path information to obtain a path render image; extracting a map element from the current location information, and rendering, according to multiple channel dimensions, the map element to a corresponding map-element render image; joining, according to the multiple channel dimensions, the path render image and the map-element render image, so as to obtain a joined image matrix; inputting the joined image matrix into a trained feature extractor, and performing feature extraction on the joined image matrix by means of the feature extractor, so as to obtain a feature extraction result; and predicting, according to the feature extraction result, paths of the multiple obstacles within a preset time period.

Description

Trajectory prediction method, device, computer equipment and storage medium

Technical field

This application relates to a trajectory prediction method, device, computer equipment and storage medium.

Background technique

The development of artificial intelligence technology has promoted the development of autonomous driving technology. In the process of automatic driving, it is very necessary to predict the trajectory of obstacles in the surrounding environment within a certain period of time. By predicting the future trajectory of the obstacle, the vehicle can recognize the intention of the obstacle earlier, and plan the driving route and speed according to the intention of the obstacle, so as to avoid collisions and reduce the occurrence of safety accidents. The future trajectory of obstacles is affected by many factors. The traditional trajectory prediction method is to predict the trajectory of the obstacle based on the obstacle's own information, and the obstacle's own information is only part of the influencing factors, resulting in low accuracy of the predicted obstacle trajectory.

Summary of the invention

According to various embodiments disclosed in the present application, a trajectory prediction method, device, computer device, and storage medium that can improve the accuracy of trajectory prediction are provided.

A trajectory prediction method, including:

Obtain historical trajectory information and current position information of multiple obstacles in the current environment;

Rendering the historical trajectory information to obtain a trajectory rendering diagram;

Extracting a map element from the current location information, and rendering the map element to a corresponding map element rendering image according to multiple channel dimensions;

Splicing the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

Input the spliced image matrix to the trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

Predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.

A trajectory prediction device includes:

The acquisition module is used to acquire historical trajectory information and current position information of multiple obstacles in the current environment;

The first rendering module is configured to render the historical trajectory information to obtain a trajectory rendering map;

The second rendering module is configured to extract map elements from the current location information, and render the map elements into corresponding map element rendering images according to multiple channel dimensions;

A splicing module, configured to splice the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

An extraction module, configured to input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

The prediction module is used to predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.

A computer device, including a memory and one or more processors, the memory stores computer readable instructions, and when the computer readable instructions are executed by the processor, the one or more processors execute The following steps:

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

The details of one or more embodiments of the present application are set forth in the following drawings and description. Other features and advantages of this application will become apparent from the description, drawings and claims.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. A person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

Fig. 1 is an application environment diagram of a trajectory prediction method in one or more embodiments.

Fig. 2 is a schematic flowchart of a trajectory prediction method in one or more embodiments.

Fig. 3 is a schematic diagram of an image matrix after stitching in one or more embodiments.

FIG. 4 is a schematic flowchart of the steps of rendering historical trajectory information to obtain a trajectory rendering diagram in one or more embodiments.

Fig. 5 is a schematic flow diagram of the steps of rendering a map element to a corresponding map element rendering diagram according to multiple channel dimensions in one or more embodiments.

Fig. 6 is a block diagram of a trajectory prediction device in one or more embodiments.

Fig. 7 is a block diagram of a computer device in one or more embodiments.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

The trajectory prediction method provided in this application can be applied to the application environment as shown in FIG. 1. The vehicle-mounted sensor 102 sends the collected information to be detected to the computer device 104. The vehicle-mounted sensor can be a lidar or a vehicle-mounted camera. The computer device 104 processes the information to be detected to obtain historical track information of multiple obstacles in the current environment. On-board computer equipment can be referred to as computer equipment for short. The vehicle locator 106 sends the collected current position information to the computer device 104. The computer device 104 renders the historical trajectory information to obtain a rendering of the trajectory. The computer device 104 extracts map elements from the current location information, and renders the map elements into corresponding map element rendering images according to multiple channel dimensions. The computer device 104 splices the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix. The computer device 104 inputs the spliced image matrix into the trained feature extractor, and performs feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result. The computer device 104 predicts the trajectory of multiple obstacles within a preset time period according to the feature extraction result.

In one of the embodiments, as shown in FIG. 2, a method for trajectory prediction is provided. Taking the method applied to the computer device in FIG. 1 as an example for description, the method includes the following steps:

Step 202: Obtain historical trajectory information and current position information of multiple obstacles in the current environment.

In the process of automatic driving of the vehicle, the on-board sensor can send the collected information to be detected to the computer device, and the computer device processes the information to be detected to obtain historical trajectory information of multiple obstacles. It is also possible to collect the information to be detected by the vehicle-mounted sensor, and then obtain the historical track information of multiple obstacles through the detector and tracker. The vehicle tracker sends the historical trajectory information of multiple obstacles to the computer device. The vehicle-mounted locator sends the collected current position information to the computer equipment. For example, the vehicle-mounted locator may be a GPS (Global Positioning System, global positioning system) locator. The vehicle tracker can receive GPS signals from satellites, analyze the GPS signals, and calculate the corresponding geographic location information, and then use GSM (Global System of Mobile communication, global mobile communication system)/CDMA (Code Division Multiple Access, code division). Multiple access) and other wireless networks send geographic location information to computer equipment.

Step 204: Render the historical trajectory information to obtain a trajectory rendering map.

The computer device renders the acquired historical trajectory information of multiple obstacles into a feature map to obtain a trajectory rendering map. The historical trajectory information may be the trajectory of each frame of the history of multiple obstacles. The computer device renders the historical trajectory information of multiple obstacles in the current frame to obtain a trajectory rendering map. The color of obstacles in each frame in the trajectory rendering diagram changes with the distance from the current frame. The farther away from the current frame, the lighter the color of the obstacle.

Step 206: Extract the map element from the current location information, and render the map element into a corresponding map element rendering image according to multiple channel dimensions.

In step 208, the trajectory rendering map and the map element rendering map are spliced according to multiple channel dimensions to obtain a spliced image matrix.

The computer equipment obtains the current position information collected by the vehicle-mounted locator. The current location information may be the location information of the vehicle on the high-precision map at the current moment. The current location information can be expressed in the form of latitude and longitude. The computer equipment extracts map elements from the current location information. Map elements can include information such as lane lines, center lines, sidewalks, and stop lines. The computer device may render the extracted map elements according to multiple channel dimensions, and render the map elements into a map element rendering map corresponding to the channel dimensions. When the map elements are different, the channel dimensions corresponding to the map elements can also be different. Channel dimensions can include color channels, element channels, and so on. The color channel can include three channels of red, green, and blue. Elemental passages can include lane-line passages, center-line passages, and sidewalk passages.

After the computer device obtains the trajectory rendering image and the map element rendering image, the trajectory rendering image and the map element rendering image can be spliced together. The computer device determines the corresponding channel dimensions of the trajectory rendering map and the map element rendering map, and performs image stitching on the trajectory rendering map and the map element rendering map in the corresponding channel dimensions to obtain a spliced image matrix. The spliced image matrix may be a complete image including the trajectory rendering map and the map element rendering map. The schematic diagram of the spliced image matrix can be shown in FIG. 3. The white circles in the figure can represent vehicles. The line formed by multiple white circles represents the trajectory of the vehicle. The lines represent lane lines. The intersection of the lines represents the center line.

In one of the embodiments, the computer device may also preprocess the trajectory rendering image and the map element rendering image before splicing the trajectory rendering image and the map element rendering image. Specifically, the computer device may perform filtering processing on the trajectory rendering image and the map element rendering image to obtain a filtered trajectory rendering image and a filtered map element rendering image. By filtering the trajectory rendering image and the map element rendering image, the computer device can obtain a smooth trajectory rendering image and map element rendering image, and can remove noise, which is beneficial to improve the accuracy of subsequent feature extraction.

Step 210: Input the spliced image matrix to the trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result.

Step 212: Perform regression prediction on the feature extraction result to obtain the trajectories of multiple obstacles within a preset time period.

The computer device has pre-trained a feature extractor before acquiring the historical trajectory information and current position information of multiple obstacles in the current environment. The feature extractor is obtained by training the convolutional neural network model according to the sample data. The feature extractor may include multiple network layer structures. For example, it can include an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

The computer equipment splices the trajectory rendering map and the map element rendering map according to multiple channel dimensions, and after the spliced image matrix is obtained, the trained feature extractor can be called, and the spliced image matrix can be input to the trained feature extractor in. The computer device extracts the image feature information and context feature information corresponding to the spliced image matrix through the feature extractor, and then outputs the feature extraction result corresponding to the spliced image matrix through the fully connected layer of the feature extractor.

The computer equipment can calculate the feature extraction results by means of regression prediction to obtain the trajectories of multiple obstacles within a preset time period. Regression prediction may be to predict the position coordinates of the obstacle in a preset time period according to the correlation or causality between the feature extraction results. The position coordinates of the obstacle at any time within the preset time period can be represented by P(x,y). For example, the preset time period may be 5s.

In this embodiment, after obtaining the historical trajectory information and current position information of many obstacles in the current environment, the computer device renders the historical trajectory information of multiple obstacles into the trajectory rendering graph, and can obtain the obstacle itself and its surroundings. The environmental information can realize the consideration of the influence factors of the trajectory from many aspects, which is more conducive to improving the accuracy of trajectory prediction. The computer device renders the map element in the current location information into the map element rendering map according to multiple channel dimensions. The current position of the obstacle can be rendered intuitively and accurately through the channel dimension corresponding to the map element, which is conducive to subsequent trajectory prediction. The computer device thus splices the trajectory rendering map and the map element rendering map according to multiple channel dimensions, and inputs the spliced image matrix into the trained feature extractor for feature extraction, and obtains the feature extraction result. It realizes the combination of various influence factors of the obstacle trajectory, and further improves the comprehensiveness of the feature extraction results. The computer equipment performs regression prediction on the feature extraction results. Since the obtained feature extraction results include the trajectory of multiple obstacles in the history frame, the scope of environmental information is expanded, and the trajectory prediction based on various influencing factors is realized, thereby providing trajectory prediction. Accuracy.

In one of the embodiments, as shown in FIG. 4, the steps of rendering historical trajectory information to obtain a trajectory rendering map include:

Step 402: Determine historical time series information according to historical track information.

In step 404, the historical timing information is merged in the current frame to obtain trajectory renderings corresponding to multiple obstacles.

The computer equipment obtains the historical trajectory information of multiple obstacles. The historical trajectory information may include the trajectory of each frame of the history of each obstacle. The computer device obtains the time corresponding to each frame of the historical trajectory according to the historical trajectory of each obstacle, and obtains the historical timing information of multiple obstacles according to the time corresponding to each frame of the historical trajectory of each obstacle. The historical timing information may include the trajectory corresponding to each frame of the history of each obstacle generated in the sequence of time. The computer device determines the rendering channel corresponding to the current frame according to the historical trajectory information, and merges the historical timing information corresponding to multiple obstacles into one image in the current frame according to the corresponding rendering channel, thereby obtaining the trajectory rendering map corresponding to the multiple obstacles .

In this embodiment, the computer device determines the historical timing information according to the historical trajectory information, and fuses the historical timing information in the current frame to obtain trajectory renderings corresponding to multiple obstacles. The ability to fuse historical time series information into an image is conducive to global analysis of the trajectory of obstacles. At the same time, there is no need to render the historical trajectory information of the obstacles into separate images one by one, which effectively saves the computing resources of the computer equipment.

In one of the embodiments, as shown in FIG. 5, the steps of rendering a map element to a corresponding map element rendering image according to multiple channel dimensions include:

Step 502: Identify whether there is a map element corresponding to the channel dimension in the map element according to each channel dimension.

In step 504, when the map element corresponding to the channel dimension exists in the map element, the map element is rendered to the map element rendering map corresponding to the channel dimension.

The computer device searches for the map element from the current location information. The current location information can be obtained in a high-precision map. Map elements can include information such as lane lines, center lines, sidewalks, and stop lines. Channel dimensions can include color channels, element channels, and so on. The color channel can include three channels of red, green, and blue. Elemental passages can include lane-line passages, center-line passages, and sidewalk passages. The computer device recognizes whether there is a map element corresponding to the channel dimension in the acquired map elements according to each channel dimension. When there is a map element corresponding to the channel dimension, the map element is rendered to the map element rendering image corresponding to the channel dimension. After all the map elements are rendered, the map information around the obstacle is obtained according to the multiple map element rendering images Rendering diagram. For example, the computer device recognizes whether there is a center line in the map element according to the center line channel, and when there is a center line, the center line in the map element is rendered to the center line rendering image corresponding to the center line channel.

In this embodiment, the computer device recognizes the corresponding map element according to each channel dimension, and renders the map element to the map element rendering diagram corresponding to the channel dimension, which can visually display the map information around the obstacle. At the same time, it is conducive to the subsequent stitching of the map element rendering map and the trajectory rendering map according to the channel dimension.

In one of the embodiments, the feature extractor includes multiple network layers, and the feature extractor is used to perform feature extraction on the stitched image matrix to obtain the feature extraction result, including: extracting the stitched image matrix through the input layer of the feature extractor The image vector and context vector in the image vector; input the image vector and context vector into the convolutional layer, extract the image feature information corresponding to the image vector and the context feature information corresponding to the context vector; input the image feature information and context feature information into the pooling layer, right Image feature information and context feature information are processed for dimensionality reduction; the image feature information after dimensionality reduction is input into the fully connected layer, and the feature extraction result corresponding to the spliced image matrix is output.

After obtaining the spliced image matrix, the computer device calls the feature extractor, and inputs the spliced image matrix into the feature extractor for feature extraction. The feature extractor is obtained by training the convolutional neural network model according to the sample data. The feature extractor may include multiple network layer structures. For example, it can include an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

The computer equipment extracts the image vector and context vector in the spliced image matrix through the input layer of the feature extractor. The input layer of the feature extractor takes the extracted image vector and context vector as the input of the convolution layer, and extracts corresponding feature information through the convolution layer to obtain image feature information and context feature information. The image feature information may include spatial feature information and time series feature information. The spatial feature information may include the historical speed change information of the obstacle. The time sequence feature information may include position information and direction information of the obstacle within a preset time period. The convolutional layer of the feature extractor thus takes the image feature information and context feature information as the input of the pooling layer, and performs dimensionality reduction processing on the image feature information and context feature information through the pooling layer. The pooling layer of the feature extractor takes the image feature information and context feature information after dimensionality reduction processing as the input of the fully connected layer, and outputs the feature extraction result corresponding to the spliced image matrix through the fully connected layer.

In this embodiment, the computer device extracts the image vector and context vector in the spliced image matrix through the input layer of the feature extractor, and extracts the image feature information corresponding to the image vector and the context feature information corresponding to the context vector through the convolutional layer, The interference information in the spliced image matrix can be filtered, and the spliced image matrix can be focused to obtain characteristic information. The computer device performs dimensionality reduction processing on the image feature information and context feature information through the pooling layer of the feature extractor, which can extract the main image feature information and context feature and avoid the influence of redundant features. The computer device then outputs the feature extraction result corresponding to the image matrix through the fully connected layer, which helps to improve the accuracy of feature extraction.

In one of the embodiments, performing regression prediction on the feature extraction result to obtain the trajectory of multiple obstacles within a preset time period includes: calculating the number of predicted points according to the preset time period and the preset sampling rate; The number of points and the feature extraction result regression predict the position change information of multiple obstacles in a preset time period; according to the position change information, the trajectory of the multiple obstacles in the preset time period is obtained.

After the computer device obtains the feature extraction result, it calculates the number of prediction points according to the preset time period and the preset sampling rate. The number of prediction points can be obtained by comparing the preset time period with the preset sampling rate. For example, if the preset time period is 5s and the preset sampling rate is 0.2, then 5/0.2=25, that is, the number of prediction points is 25. The computer equipment regression predicts the location information of each predicted point according to the feature extraction result. The location information may be the location coordinates of the obstacle. The computer device calculates the position change information of the obstacle within the preset time period according to the position information of the multiple prediction points. The position change information may be a position offset. The computer device further obtains the trajectory of the obstacle within the preset time period according to the position change information.

In this embodiment, the computer device calculates the number of prediction points according to the preset time period and the preset sampling rate, and predicts the position change information of multiple obstacles within the preset time period based on the number of prediction points and the feature extraction result. , And then obtain the trajectories of multiple obstacles within a preset time period according to the position change information. Since the feature extraction result contains feature information of multiple obstacles, it provides a wider range of context feature information. At the same time, the trajectory of multiple obstacles within a preset time period can be obtained by only one prediction, thereby effectively reducing the amount of calculation, improving the efficiency of trajectory prediction, and realizing real-time trajectory prediction of obstacles.

In one of the embodiments, obtaining historical trajectory information of obstacles in the current environment includes: obtaining information to be detected; detecting the information to be detected according to the type of information to be detected, and determining the obstacle in the current environment; and moving the obstacle The process is tracked and the historical trajectory information of obstacles is obtained.

In the process of automatic driving, the vehicle-mounted sensor collects the information to be detected, and sends the collected information to be detected to the computer equipment. The computer equipment detects the information to be detected according to the type of the information to be detected, and determines the obstacle information in the environment at the current moment. The vehicle-mounted sensor can be a lidar or a vehicle-mounted camera. When the vehicle-mounted sensor is a lidar, the type of information to be detected is point cloud data. The computer equipment can classify the point cloud data to determine the obstacles in the environment at the current moment. When the vehicle-mounted sensor is a vehicle-mounted camera, the type of information to be detected is an image. Computer equipment can segment and semantically label images according to semantic categories, and determine obstacles in the current environment. The computer equipment tracks the movement process of the obstacle, predicts the current position information based on the position information of the obstacle at the previous time, and compares the predicted current position information with the actual position information to obtain error information. According to the error information, the position information at the next moment is corrected to obtain the historical trajectory information of multiple obstacles. Computer equipment can obtain multiple types of information to be detected, determine the corresponding detection method according to the type of information to be detected, realize the detection of obstacles in the current environment, track the movement process of the obstacles, and obtain the history of the obstacles Track information. It can flexibly detect obstacles and obtain corresponding historical trajectory information.

In one of the embodiments, as shown in FIG. 6, a trajectory prediction device is provided, including: an acquisition module 602, a first rendering module 604, a second rendering module 606, a splicing module 608, an extraction module 610, and a prediction module 612 ,among them:

The obtaining module 602 is used to obtain historical trajectory information and current position information of multiple obstacles in the current environment.

The first rendering module 604 is configured to render historical trajectory information to obtain a trajectory rendering map.

The second rendering module 606 is configured to extract map elements from the current location information, and render the map elements into corresponding map element rendering images according to multiple channel dimensions.

The splicing module 608 is used for splicing the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix.

The extraction module 610 is configured to input the spliced image matrix to the trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result.

The prediction module 612 is configured to predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.

In one of the embodiments, the first rendering module 604 is further configured to determine historical time series information according to historical trajectory information; merge the historical time series information in the current frame to obtain trajectory rendering maps corresponding to multiple obstacles.

In one of the embodiments, the second rendering module 606 is further configured to identify whether there is a map element corresponding to the channel dimension in the map element according to each channel dimension; when there is a map element corresponding to the channel dimension in the map element, the map The element is rendered to the map element rendering image corresponding to the channel dimension.

In one of the embodiments, the extraction module 610 is also used to extract the image vector and the context vector in the stitched image matrix through the input layer of the feature extractor; input the image vector and the context vector into the convolutional layer, and extract the corresponding image vector Image feature information and context feature information corresponding to the context vector; input image feature information and context feature information to the pooling layer, and perform dimensionality reduction processing on image feature information and context feature information; image feature information and context feature after dimensionality reduction processing The information is input to the fully connected layer, and the feature extraction result corresponding to the spliced image matrix is output.

In one of the embodiments, the prediction module 612 is further configured to calculate the number of prediction points according to the preset time period and the preset sampling rate; according to the number of prediction points and the feature extraction result, regression prediction of multiple obstacles in the preset time period The position change information within; obtain the trajectory of multiple obstacles in a preset time period according to the position change information.

In one of the embodiments, the acquisition module 602 is also used to acquire the information to be detected; to detect the information to be detected according to the type of the information to be detected, to determine the obstacles in the current environment; to track the movement process of the obstacles to obtain the obstacles Historical track information.

For the specific limitation of the trajectory prediction device, please refer to the above limitation on the trajectory prediction method, which will not be repeated here. Each module in the above-mentioned trajectory prediction device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one of the embodiments, a computer device is provided, and its internal structure diagram may be as shown in FIG. 7. The computer equipment includes a processor, a memory, a communication interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer equipment is used to store the historical trajectory information and current position information of obstacles. The communication interface of the computer device is used to connect and communicate with the vehicle-mounted sensor and the vehicle-mounted locator. The computer readable instructions are executed by the processor to realize a trajectory prediction method.

Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

A computer device that includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute each of the foregoing method implementations. The steps in the example.

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps in each of the foregoing method embodiments. step.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

A trajectory prediction method, including:

Obtain historical trajectory information and current position information of multiple obstacles in the current environment;

Rendering the historical trajectory information to obtain a trajectory rendering diagram;

Extracting a map element from the current location information, and rendering the map element to a corresponding map element rendering image according to multiple channel dimensions;

Splicing the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

Input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

Predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.
The method according to claim 1, wherein the rendering the historical trajectory information to obtain a trajectory rendering map comprises:

Determine historical time sequence information according to the historical track information; and

The historical time sequence information is merged in the current frame to obtain a trajectory rendering map corresponding to multiple obstacles.
The method according to claim 1, wherein the rendering the map element to a corresponding map element rendering image according to multiple channel dimensions comprises:

Identify whether there is a map element corresponding to the channel dimension in the map element according to each channel dimension; and

When the map element corresponding to the channel dimension exists in the map element, the map element is rendered to the map element rendering map corresponding to the channel dimension.
The method according to claim 1, wherein the feature extractor includes a plurality of network layers, and the feature extractor is used to perform feature extraction on the stitched image matrix to obtain a feature extraction result, including :

Extracting the image vector and the context vector in the spliced image matrix through the input layer of the feature extractor;

Inputting the image vector and the context vector into the convolutional layer, and extracting the image feature information corresponding to the image vector and the context feature information corresponding to the context vector;

Input the image feature information and context feature information to the pooling layer, and perform dimensionality reduction processing on the image feature information and context feature information; and

The image feature information and context feature information after the dimensionality reduction process are input into the fully connected layer, and the feature extraction result corresponding to the spliced image matrix is output.
The method according to claim 1, wherein the predicting the trajectory of a plurality of obstacles within a preset time period according to the feature extraction result comprises:

Calculate the number of prediction points according to the preset time period and preset sampling rate;

According to the number of prediction points and the feature extraction result, regression prediction of position change information of multiple obstacles within a preset time period; and

Obtain the trajectories of multiple obstacles within a preset time period according to the position change information.
The method according to any one of claims 1 to 5, wherein the acquiring historical trajectory information of obstacles in the current environment comprises:

Obtain the information to be detected;

Detect the information to be detected according to the type of the information to be detected, and determine obstacles in the current environment; and

The movement process of the obstacle is tracked, and the historical trajectory information of the obstacle is obtained.
A trajectory prediction device includes:

The acquisition module is used to acquire historical trajectory information and current position information of multiple obstacles in the current environment;

The first rendering module is configured to render the historical trajectory information to obtain a trajectory rendering map;

The second rendering module is configured to extract map elements from the current location information, and render the map elements into corresponding map element rendering images according to multiple channel dimensions;

A splicing module, configured to splice the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

An extraction module, configured to input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

The prediction module is used to predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.
8. The device according to claim 7, wherein the first rendering module is further configured to determine historical timing information according to the historical trajectory information; and fusing the historical timing information in the current frame to obtain multiple obstacles The trajectory rendering map corresponding to the object.
7. The device according to claim 7, wherein the second rendering module is further configured to identify in the map element whether there is a map element corresponding to the channel dimension according to each channel dimension; and when the map When the map element corresponding to the channel dimension exists in the element, the map element is rendered to the map element rendering map corresponding to the channel dimension.
7. The device according to claim 7, wherein the extraction module is further configured to extract the image vector and the context vector in the stitched image matrix through the input layer of the feature extractor; And the context vector input convolutional layer, extract the image feature information corresponding to the image vector and the context feature information corresponding to the context vector; input the image feature information and the context feature information into the pooling layer, and compare the image feature information Perform dimensionality reduction processing with the context feature information; and input the dimensionality reduction processed image feature information and context feature information into the fully connected layer, and output the feature extraction result corresponding to the spliced image matrix.
A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the one or more processors Each processor performs the following steps:

Obtain historical trajectory information and current position information of multiple obstacles in the current environment;

Rendering the historical trajectory information to obtain a trajectory rendering diagram;

Extracting a map element from the current location information, and rendering the map element to a corresponding map element rendering image according to multiple channel dimensions;

Splicing the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

Input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

Predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.
The computer device according to claim 11, wherein the processor further executes the following steps when executing the computer-readable instructions: determining historical timing information according to the historical trajectory information; and storing the historical timing information in The current frame is fused to obtain the trajectory rendering map corresponding to multiple obstacles.
The computer device according to claim 11, wherein the processor further executes the following step when executing the computer-readable instructions: identifying in the map element whether there is a corresponding channel dimension according to each channel dimension And when the map element corresponding to the channel dimension exists in the map element, rendering the map element to the map element rendering map corresponding to the channel dimension.
The computer device according to claim 11, wherein the processor further executes the following step when executing the computer-readable instruction: extracting the images in the spliced image matrix through the input layer of the feature extractor Image vector and context vector; input the image vector and context vector to the convolutional layer, extract the image feature information corresponding to the image vector and the context feature information corresponding to the context vector; combine the image feature information and context feature information Input the pooling layer to perform dimensionality reduction processing on the image feature information and context feature information; and input the dimensionality reduction processed image feature information and context feature information into the fully connected layer, and output the features corresponding to the spliced image matrix Extract the results.
The computer device according to claim 11, wherein the processor further executes the following steps when executing the computer-readable instructions: calculating the number of prediction points according to a preset time period and a preset sampling rate; The number of prediction points and the feature extraction result regression predict the position change information of the multiple obstacles in a preset time period; and obtain the trajectory of the multiple obstacles in the preset time period according to the position change information.
One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Obtain historical trajectory information and current position information of multiple obstacles in the current environment;

Rendering the historical trajectory information to obtain a trajectory rendering diagram;

Extracting a map element from the current location information, and rendering the map element to a corresponding map element rendering image according to multiple channel dimensions;

Splicing the trajectory rendering map and the map element rendering map according to multiple channel dimensions to obtain a spliced image matrix;

Input the spliced image matrix to a trained feature extractor, and perform feature extraction on the spliced image matrix through the feature extractor to obtain a feature extraction result; and

Predict the trajectory of multiple obstacles in a preset time period according to the feature extraction result.
The storage medium according to claim 16, wherein when the computer-readable instructions are executed by the processor, the following steps are further performed: determining historical timing information according to the historical trajectory information; and converting the historical timing information Perform fusion in the current frame to obtain the trajectory rendering map corresponding to multiple obstacles.
The storage medium according to claim 16, characterized in that, when the computer-readable instructions are executed by the processor, the following step is further performed: identifying whether the channel dimension exists in the map element according to each channel dimension Corresponding map element; and when the map element corresponding to the channel dimension exists in the map element, rendering the map element to the map element rendering map corresponding to the channel dimension.
The storage medium according to claim 16, wherein, when the computer-readable instructions are executed by the processor, the following step is further executed: extracting from the spliced image matrix through the input layer of the feature extractor The image vector and context vector of the image vector; input the image vector and the context vector into the convolutional layer, and extract the image feature information corresponding to the image vector and the context feature information corresponding to the context vector; combine the image feature information and the context feature The information is input to the pooling layer to perform dimensionality reduction processing on the image feature information and context feature information; and the image feature information and context feature information after the dimensionality reduction process are input to the fully connected layer, and the corresponding image matrix is output Feature extraction results.
The storage medium according to claim 16, wherein when the computer-readable instructions are executed by the processor, the following steps are further executed: calculating the number of prediction points according to a preset time period and a preset sampling rate; The number of prediction points and the feature extraction result regression predict the position change information of the multiple obstacles in a preset time period; and obtain the trajectory of the multiple obstacles in the preset time period according to the position change information.