WO2020191642A1 - 轨迹预测方法及装置、存储介质、驾驶系统与车辆 - Google Patents

轨迹预测方法及装置、存储介质、驾驶系统与车辆 Download PDF

Info

Publication number
WO2020191642A1
WO2020191642A1 PCT/CN2019/079780 CN2019079780W WO2020191642A1 WO 2020191642 A1 WO2020191642 A1 WO 2020191642A1 CN 2019079780 W CN2019079780 W CN 2019079780W WO 2020191642 A1 WO2020191642 A1 WO 2020191642A1
Authority
WO
WIPO (PCT)
Prior art keywords
global
trajectory
data
semantic
predicted
Prior art date
Application number
PCT/CN2019/079780
Other languages
English (en)
French (fr)
Inventor
崔健
陈晓智
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2019/079780 priority Critical patent/WO2020191642A1/zh
Priority to CN201980005403.6A priority patent/CN111316286A/zh
Publication of WO2020191642A1 publication Critical patent/WO2020191642A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights

Definitions

  • the present invention relates to the field of intelligent transportation technology, in particular to a trajectory prediction method and device, storage medium, driving system and vehicle.
  • the prediction algorithm for the trajectory of moving objects has great significance in the field of path planning.
  • path planning can be performed when the possible future motion trajectory of the moving object is known, which is beneficial to prevent accidents such as collisions.
  • Current trajectory prediction algorithms are generally based on the motion data of the moving object itself, determine the applicable motion model of the moving object according to the category of the moving object, and use the motion model to process the motion data of the moving object itself, and then post-processing Integrating regional semantic information can predict the motion trajectory of the object to be predicted.
  • the existing trajectory prediction algorithm is based on the motion data of the moving object itself, and cannot predict the trajectory from the global perspective. This easily leads to the intersection of the predicted trajectories of different moving objects, which may lead to collisions in path planning or scheduling based on this. Unexpected accidents have major safety hazards.
  • the embodiments of the present invention provide a trajectory prediction method and device, storage medium, driving system, and vehicle, which can realize the trajectory prediction of moving objects in combination with global data, have high prediction accuracy, and reduce accidents to a certain extent. Probability of occurrence.
  • an embodiment of the present invention provides a trajectory prediction method, including:
  • the trained trajectory prediction model is used to process the global feature to obtain the target trajectory of the object to be predicted.
  • an embodiment of the present invention provides a trajectory prediction device, including:
  • An acquisition module for acquiring global semantic data and global trajectory data of the area where the object to be predicted is located;
  • a fusion module for fusing the global semantic data and the global trajectory data to obtain global fusion data
  • the feature extraction module is used to extract features in the global fusion data to obtain global features
  • the prediction module is used to process the global feature using the trained trajectory prediction model to obtain the target trajectory of the object to be predicted.
  • an embodiment of the present invention provides a trajectory prediction device, including:
  • the computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.
  • an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored
  • the computer program is executed by the processor to implement the method according to the first aspect.
  • an embodiment of the present invention provides a driving system, including:
  • a trajectory prediction device for executing the method described in the first aspect
  • the motion controller is used to control the motion of the controlled object according to the target trajectory.
  • the controlled object and the to-be-predicted object are different objects.
  • an embodiment of the present invention provides a vehicle, including:
  • the trajectory prediction device according to the second aspect or the third aspect is used to execute the method according to the first aspect.
  • an embodiment of the present invention provides a vehicle, including:
  • an embodiment of the present invention provides a control device for an unmanned aerial vehicle, including:
  • the technical solution provided by the embodiment of the present invention can obtain and process the global semantic data and global trajectory data of the region where the object to be predicted is located, so that the global features of the region can be obtained, and further, the trained trajectory prediction model is used to perform the global features Through processing, the target trajectory of the object to be predicted can be obtained.
  • the technical solution provided by the embodiment of the present invention starts from the global semantics and global trajectory.
  • predicting the trajectory of an object to be predicted all moving objects in the area are considered.
  • the trajectory prediction of any moving object in the area can be realized.
  • this solution has a higher prediction accuracy, and when Using this as a basis for subsequent path planning or scheduling can also reduce the probability of accidents to a certain extent and have higher safety.
  • FIG. 1 is a schematic top view of a trajectory prediction scene provided by an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a trajectory prediction method provided by an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another trajectory prediction method provided by an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of another method for trajectory prediction according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a recurrent unit structure of a recurrent neural network model provided by an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of the model architecture of a recurrent neural network model provided by an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of the model architecture of a long short-term memory network model provided by an embodiment of the present invention.
  • FIG. 8 is a schematic flowchart of another trajectory prediction method provided by an embodiment of the present invention.
  • FIG. 9 is a functional block diagram of a trajectory prediction device provided by an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of the physical structure of a trajectory prediction device provided by an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a driving system provided by an embodiment of this aspect.
  • FIG. 12 is a schematic diagram of the architecture of a vehicle provided by an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of another vehicle provided by an embodiment of the present invention.
  • Moving object refers to a creature or object that can move on a trajectory.
  • the moving objects involved in the embodiments of the present invention may include, but are not limited to, at least one of vehicles, animals, humans, robots, and unmanned aerial vehicles.
  • the vehicle may be an unmanned vehicle, such as an unmanned ground vehicle (UGV), or a private vehicle or a public bus in an automatic driving mode.
  • UUV unmanned ground vehicle
  • Object to be predicted One or more moving objects of the target track to be predicted. Among them, multiple refers to two or more than two, and this concept is involved in the following, and will not be repeated.
  • Semantic objects refer to the objects of each semantic concept in the area. Please refer to the scene shown in Figure 1.
  • the semantic objects in this scene include: vehicles, lane lines, and lanes. It can be seen that Fig. 1 is only used as an example.
  • the semantic categories of semantic objects also include a variety of semantic categories, such as trees, obstacles, railings, signs, people, animals, etc.
  • the embodiments of the present invention are specific to each The semantic category of semantic objects is not particularly limited.
  • LSTM Long Short Term Memory
  • RNN Recurrent Neural Network
  • the specific application scenario of the technical solution provided by the embodiment of the present invention is: a trajectory prediction scenario for a moving object.
  • the technical solution provided by the embodiment of the present invention can also be specifically applied to a path planning scenario.
  • the path planning for one or more moving objects can be realized according to the predicted trajectory.
  • the technical solutions provided by the embodiments of the present invention may also be specifically applied to vehicle scheduling scenarios. For example, by predicting the trajectory of other unschedulable vehicles or objects, the scheduling of schedulable vehicles can be realized.
  • the existing trajectory prediction method is only for a single moving object. After the category of the moving object is determined, the motion data of the moving object itself is processed through the motion model corresponding to the category. Thus, the trajectory of the moving object is predicted.
  • this prediction method is subject to the restriction of the object category. It is necessary to accurately determine the category of the moving object in order to obtain a more accurate prediction result with the motion model corresponding to the category; on the other hand, this prediction method only depends on The motion data of the moving object itself has not been comprehensively analyzed with the current motion environment and the motion of other moving objects from a global perspective.
  • the technical solution provided by the embodiment of the present invention aims to solve the above technical problems of the prior art, and proposes the following solution idea: comprehensively consider the global data of the region where the object to be predicted is located, including global semantic data and global trajectory data, according to This obtains the global feature, and uses the global feature as the input of the trajectory prediction model to obtain the target trajectory of the object to be predicted.
  • the trajectory prediction method provided by the embodiments of the present invention can be specifically executed in a built-in processor or terminal device held by a certain moving object, or it can also be specifically executed in a cloud or a background server.
  • the first processor of the self-driving vehicle can plan the driving route by itself, and the second processor of the self-driving vehicle is used to execute the trajectory prediction method provided by this solution, and is used to calculate the predicted trajectory Input the first built-in processor so that the first processor can perform subsequent path planning according to the predicted trajectory.
  • the first processor and the second processor may be the same processor, or may also be different processors, for example, may be one or two processors in an Advanced Driving Assistant System (ADAS) ;
  • ADAS Advanced Driving Assistant System
  • the first processor and the second processor may be part of the overall controller of the vehicle, or may also be a background overall server or cloud server that controls the driving of the unmanned vehicle.
  • the embodiment of the present invention provides a trajectory prediction method. Please refer to Figures 2 to 4, where Figure 2 shows a schematic flow chart of a trajectory prediction method provided by an embodiment of the present invention, and Figure 3 shows how the trajectory prediction method provided by an embodiment of the present invention works in a specific A schematic diagram of the implementation process in the application scenario, and Figure 4 is a specific implementation of the process shown in Figure 3.
  • the method includes the following steps:
  • S202 Obtain global semantic data and global trajectory data of the region where the object to be predicted is located.
  • the object to be predicted is a moving object, which may include but is not limited to at least one of the following: vehicles, animals, humans, robots, and unmanned aerial vehicles.
  • the number of objects to be predicted in the embodiment of the present invention is not particularly limited, and may be one or more.
  • the global semantic data is used to describe the semantic category of each object in the area where the object to be predicted is located.
  • each moving object or non-moving object is regarded as a semantic object and has its own semantic category.
  • Global trajectory data is used to describe the historical movement coordinates of each moving object in the area. It can be seen that in specific implementation, the global trajectory data may include at least one frame of historical motion coordinates.
  • Figure 3 shows a schematic diagram of the implementation process of this solution when the scenario shown in Figure 1 is taken as an example.
  • the driving scene contains three semantics, namely: vehicle, lane and lane line.
  • the global semantic data that needs to be obtained in the scene that is, the semantic data of each semantic object; and
  • the global trajectory data of the scene that is, the trajectory data of each moving object: vehicle 1 (as the object to be predicted), vehicle 2 and vehicle 3.
  • FIG. 3 is only schematic.
  • the expression form of global trajectory data and global semantic data is not limited to a single acquisition method, and can be acquired as a whole data.
  • the trajectory data of each moving object can be implemented by an LSTM model, which will be described in detail later.
  • the fusion module is used to fuse the global semantic data corresponding to each frame with the global trajectory data to obtain the global fusion data of each frame.
  • the feature extraction module is used to extract global features from the global fusion data.
  • this step can be implemented by a Convolutional Neural Network (Convolutional Neural Networks, CNN) model, which will be described in detail later.
  • CNN Convolutional Neural Networks
  • S208 Use the trained trajectory prediction model to process the global feature to obtain the target trajectory of the object to be predicted.
  • the present invention uses a trained trajectory prediction model to predict the motion trajectory of the moving object.
  • the input of the trajectory prediction model is the global feature
  • the output is the motion trajectory of the moving object.
  • the trajectory prediction model provided by the embodiment of the present invention may include, but is not limited to: at least one of the following: LSTM model, Multi-Layer Perception (MLP); wherein, the multilayer perception includes : RNN model, gated recurrent neural network (GRU) model.
  • LSTM model Multi-Layer Perception
  • MLP Multi-Layer Perception
  • GRU gated recurrent neural network
  • the trajectory prediction is realized by the LSTM model.
  • Y LSTM (feature), where Y represents the target trajectory, and feature represents the global feature.
  • Figure 5 shows the design logic of a cyclic unit in the RNN model
  • Figure 6 shows a schematic diagram of the model architecture of the RNN model
  • Figure 7 shows a schematic diagram of the model architecture of the LSTM model.
  • RNN model as an effective means of time series modeling, compared with ordinary neural networks, as shown in Figure 5, the main difference lies in the output of the previous frame or the intermediate state as the input of the current frame, in order to achieve historical information Integration of timely and orderly relationships.
  • the RNN model After unfolding the cyclic unit as shown in Fig. 5 in time, the RNN model as shown in Fig. 6 can be obtained.
  • the RNN model can implement timing modeling. Therefore, the trajectory prediction step in this solution can be realized through the RNN model.
  • each repeating module in the LSTM model contains 4 interaction layers, which interact in a special way, so that the output or intermediate state of the previous frame is used as the input of the current frame. Therefore, compared to the RNN model shown in Fig. 6, the LSTM model has better time-dependent modeling capabilities. In addition, considering that the prediction of the trajectory and the historical data of the moving object have a strong time correlation, the use of the LSTM model to realize the trajectory prediction can obtain a trajectory prediction result closer to the actual development.
  • the technical solution provided by the embodiment of the present invention can realize the trajectory prediction of any object to be predicted based on global data. Compared with the trajectory prediction method that only starts from the motion data of the object to be predicted, this solution It has a high prediction accuracy, and when it is used as a basis for subsequent path planning or scheduling, it can also reduce the probability of accidents to a certain extent and have higher safety.
  • S202 includes the acquisition of global data in two aspects: global semantic data and global trajectory data. Refer to the process shown in FIG. 8 for the acquisition methods of these two global data.
  • the method of obtaining global semantic data may include the following steps:
  • S202-12 Obtain a global area image of the area where the object to be predicted is located.
  • the global area image can be an image obtained in real time, and the image obtained in real time has higher timeliness, and the global semantic data obtained thereby is more accurate.
  • the real-time acquisition method may include, but is not limited to: real-time acquisition of images through an image acquisition device.
  • the image acquisition device may be a part of the trajectory prediction device (the execution body of the trajectory prediction method); or, it can have real-time data interaction with the trajectory prediction device.
  • the trajectory prediction device can be the processor A in the main controller of the vehicle
  • the image acquisition device can be a camera in the black box of the vehicle, which can directly input the collected images to the processor A; or, image acquisition
  • the device may be a camera set in the area, such as a camera set on a road or a roadside.
  • the processor A may request and receive the global area image from the camera in the area or the background server of the camera in the area in real time.
  • global semantic data can also be acquired by calling the collected data.
  • the global area image of the area in the high-resolution map can be acquired; in another implementation scenario, the global area image of the area that has been collected in other processors or memories can be acquired.
  • this implementation can only obtain the environmental information of the area, such as non-real-time data of non-moving objects such as roads, signs, lane lines, etc., but cannot obtain the movement of moving objects in the real-time scene. Therefore, in When the solution is implemented in this way, it is only suitable for trajectory prediction for a single object to be predicted, and cannot be combined with the trajectories of other moving objects to achieve comprehensive prediction, and the prediction accuracy rate is weak.
  • the image to be processed subsequently in the embodiment of the present invention is a top view image. Therefore, if the captured image is not a top view image obtained by the foregoing implementation, it is also necessary to perform a top view projection of the captured image to satisfy the subsequent processing. The required top view image.
  • the top view image involved in the embodiment of the present invention may be specifically represented as a digital orthophoto (Digital Orthophoto Map, DOM) image.
  • digital orthophoto Digital Orthophoto Map, DOM
  • the area shape or size of the "area where the object to be predicted is located" may be further personalized, which is not particularly limited in the embodiment of the present invention.
  • a rectangular area with a certain length and width in the top view can be obtained with the object to be predicted as the center.
  • a rectangular area with a length of W and a width of H as shown in Figure 1 (or Figure 3) can be obtained.
  • Image For another example, the entire road where the object to be predicted is located can also be used as the area where the object to be predicted is located.
  • S202-14 Perform semantic recognition on each pixel in the global area image to obtain the semantic category of each pixel.
  • the semantic recognition of each pixel can be realized through deep learning. If implemented in this way, it is necessary to use the preset pixel sample data to perform in-depth learning of the pixel semantic recognition model to obtain a pixel semantic recognition model that meets the application requirements (which can be achieved through the definition of the loss function) . In this way, when performing this step, only the global area image needs to be input to the pixel semantic recognition model, and the output of the pixel semantic recognition model is the semantic category of each pixel.
  • the pixel value of each pixel can also be based on the pixel value of each pixel and the pixel interval corresponding to each semantic category are compared, so that for any pixel value, the pixel value falls into
  • the semantic category corresponding to the pixel interval of is used as the semantic category corresponding to the pixel.
  • the correspondence between each semantic category and the pixel interval can be preset in a custom way.
  • S202-16 Perform semantic annotation on the global area image according to the semantic category of each pixel to obtain the global semantic information.
  • the global area image can be semantically segmented according to the semantic category of each pixel to obtain multiple semantic objects; thus, semantic annotations are performed on each semantic object separately , To obtain the global semantic information.
  • semantic labeling is only used to distinguish the semantic category of each object, and can be labelled in any distinguishable manner.
  • each semantic object can be identified by different colors. Or, as shown in FIG. 1 (or FIG. 3), different shadings may be used to identify each semantic object. It can be seen that after identification, semantic objects with the same identification are the same type of semantic objects.
  • the labeling manner of other moving objects of the same category as the object to be predicted may be the same as or different from the labeling manner of the object to be predicted.
  • the object to be predicted is vehicle 1
  • the area where the object to be predicted is located also includes moving objects of the same category: vehicle 2 and vehicle 3.
  • the first trajectory prediction model (the first trajectory prediction model is used to predict the target trajectory of the object to be predicted, the implementation method will be described in detail later) to implement step S208, as shown in FIG.
  • the identification is distinguished from the vehicle 2 and the vehicle 3.
  • the vehicle 1 has one type of identification, and the vehicle 2 and the vehicle 3 are another type of identification.
  • the second trajectory prediction model (the second trajectory prediction model is used to predict the motion trajectories of all moving objects in the area, the implementation method will be described in detail later) to implement the step S208, there is no need to perform S208 on the moving objects of the same category.
  • the vehicle 1, the vehicle 2 and the vehicle 3 can be identified using the same identification method (the same identification method is not shown in Figure 1).
  • each grid may include one or more pixels, and the division method can be realized by a preset resolution.
  • the global area image shown in FIG. 1 can be divided into a square grid with a length of 20 cm. In this way, when performing subsequent semantic annotation, only the grid needs to be annotated.
  • the implementation of the grid division is beneficial to reduce the amount of marks and improve the processing efficiency.
  • steps S202-14 and S202-16 described in FIG. 8 can also be implemented by a neural network model. That is, before performing S202-14, train the semantic recognition model, so that the global area image obtained in step S202-12 is input into the semantic recognition model, and the output of the semantic recognition model is the global area image marked with semantic categories , And get the global semantic data.
  • CNN models or other neural network models can be used.
  • the sample data of the two needs to be labeled according to the input and output of the model. Design, no longer go into details.
  • the method of obtaining global trajectory data may include the following steps:
  • S202-22 Obtain a set of trajectory points of each moving object in the area where the object to be predicted is located, where the set of trajectory points is formed by collecting coordinate points of the moving object in a time series sequence.
  • This step is used to obtain the track point set of each moving object in the current area, where each track point set is composed of multiple coordinate points of the moving object.
  • the coordinate points of each moving object can be converted to the same A coordinate point in a coordinate system.
  • the coordinate points in each track point concentration can be converted into coordinate points in a rectangular coordinate system formed by two rectangular sides of the rectangular area shown in Fig. 1, and the expression form of each coordinate point is (X, Y),
  • the trajectory point set of each moving object can be expressed as ⁇ (X i , Y i ) ⁇ , where i is used to indicate the sequence of coordinate points.
  • the aforementioned coordinate point set can be formed by acquiring the coordinate points of each moving object in a time interval with the current moment as the end point.
  • the length of the time interval can be preset according to needs, for example, a set of trajectory points of each moving object within 3s before the current time can be obtained.
  • the track point set may be actively monitored by the execution subject, or may be obtained by requesting data from other processors or collection devices.
  • the execution subject is the processor A in the main controller of the vehicle 1
  • the coordinate points of the vehicle 1 can be collected by its own locator, such as GPS, and the coordinate data collected by the locator of the vehicle 1 Send to processor A, and processor A will perform coordinate conversion to obtain the trajectory point set of own vehicle 1.
  • the trajectory point set of other vehicles can be obtained by requesting other processors, for example, if there is communication with other vehicles , You can obtain the trajectory point set from other vehicles respectively; for example, you can obtain the trajectory point set of other vehicles from the road monitor in the area; in addition, you can also collect images of other vehicles by yourself and calculate the distance from yourself , Calculate the track point set of other vehicles. There can be multiple implementation methods, which will not be repeated here.
  • the data source (or direct collection source) of the track point set may be different from the data source (or direct collection source) of the global area image in S202-12.
  • This step can also be implemented by a neural network model.
  • the trajectory point set of each moving object obtained in S202-22 (for example, the trajectory point set of each moving object within 3s) is input into the coding model (a trained neural network).
  • the model for example, the LSTM model shown in FIG. 4 can be used, and the output of the encoding model is the trajectory feature (encoder) of each moving object.
  • the length of the trajectory feature (encoder) can be assumed to be C, and the value of C is generally a preset empirical value.
  • S202-26 Construct a trajectory tensor according to the trajectory characteristics of each moving object to serve as the global trajectory data.
  • this step can construct a trajectory tensor (tensor) with a size of C*H*W, and store the trajectory features (encoder) of each moving object in This tensor is fine.
  • the encoder of the moving object is correspondingly stored in the center position of the moving object.
  • the grid can also be divided in the tensor in the manner shown in Figure 1. In this way, the center position of the moving object is located in which grid in the Tensor, the encoder of the moving object is correspondingly stored Just in the grid.
  • the global semantic data and global trajectory data can be acquired.
  • the global semantic data can be represented as a W*H image
  • the global trajectory data can be represented as a tensor of size C*H*W. Therefore, the fusion step described in S204 is performed At this time, the two can be fused into a fusion tensor with a size of (C+1)*H*W.
  • the fusion tensor can be specifically expressed as: tensor((C+1)*H*W).
  • the feature extraction model involved in the embodiment of the present invention at least includes: a convolutional neural network (Convolutional Neural Networks, CNN) model as shown in FIG. 4. Similar to the aforementioned method of implementing data processing using a neural network model, it is necessary to train the CNN model using feature extraction samples before performing this step. The model training process will not be repeated.
  • CNN convolutional Neural Networks
  • trajectory prediction model (and the neural network model involved in the foregoing implementation methods) is generally completed before the implementation of this solution, so as to achieve real-time and efficient trajectory prediction.
  • This implementation method has Higher real-time performance is conducive to real-time trajectory prediction, and then to realize path planning or scheduling in near real-time.
  • the first trajectory prediction model that predicts a single moving object (object to be predicted) based on global features can be trained.
  • this single prediction method has faster processing efficiency, which is conducive to path planning and scheduling in real-time scenarios.
  • a second trajectory prediction model that predicts all moving objects contained in the region based on the global feature.
  • the global features are input to the second trajectory prediction model, and the motion trajectories of all moving objects in the area output by the second trajectory prediction model are obtained, and The motion trajectory of the object to be predicted among all the moving objects may be used as the target trajectory.
  • This global prediction method can output the motion trajectories of all moving objects in the area at one time, which is conducive to the realization of global scheduling, and is also conducive to reducing the probability of accidents during scheduling or path planning, and improving safety.
  • a third trajectory prediction model based on multiple (not all) moving objects contained in the global feature prediction area can also be trained.
  • the model training method and the implementation of S208 are the same as above, and will not be repeated.
  • the technical solutions provided by the embodiments of the present invention can not only realize the trajectory prediction for a single moving object, but also realize the trajectory prediction for multiple moving objects.
  • the trajectory prediction model described above is independent of the object category.
  • the trajectory prediction of any type of moving object can be realized, which has higher flexibility and is also suitable for objects with multiple types The trajectory prediction in the scene.
  • the sample data is related data of the type of objects.
  • the target trajectory of the object to be predicted is obtained through the foregoing implementation methods, the target trajectory can be used for further processing.
  • motion planning may be performed for the object to be predicted according to the target trajectory. That is, further path planning is realized according to the predicted trajectory of the object to be predicted.
  • motion planning can be performed for other moving objects according to the target trajectory. That is, in the process of path planning for one or more other moving objects, the route can be planned according to the predicted trajectory of the object to be predicted to avoid collisions or other safety accidents with the object to be predicted.
  • the embodiment of the present invention further provides an embodiment of an apparatus that implements each step and method in the foregoing method embodiment.
  • the embodiment of the present invention provides a trajectory prediction device. Please refer to FIG. 9.
  • the trajectory prediction device 600 includes:
  • the obtaining module 61 is used to obtain global semantic data and global trajectory data of the area where the object to be predicted is located;
  • the fusion module 62 is configured to fuse the global semantic data and the global trajectory data to obtain global fusion data;
  • the feature extraction module 63 is configured to extract features in the global fusion data to obtain global features
  • the prediction module 64 is configured to use the trained trajectory prediction model to process the global feature to obtain the target trajectory of the object to be predicted.
  • the acquisition module 61 is specifically used for:
  • the global region image is semantically annotated to obtain the global semantic information.
  • the obtaining module 61 is further specifically configured to:
  • Semantic annotation is performed on each semantic object to obtain the global semantic information.
  • the global area image involved in the embodiment of the present invention is a digital orthophoto DOM image.
  • the acquisition module 61 is specifically used for:
  • a trajectory tensor is constructed as the global trajectory data.
  • the fusion module 63 is specifically used for:
  • the feature extraction model involved in the embodiment of the present invention at least includes: a convolutional neural network CNN model.
  • the trajectory prediction model involved in the embodiment of the present invention includes at least one of the following: a long and short-term memory network LSTM model and a multilayer perceptron MLP;
  • the multilayer perceptron includes: a recurrent neural network RNN model and a gated recurrent unit GRU model.
  • the trajectory prediction model is a first trajectory prediction model, and the first trajectory prediction model is used to predict the target trajectory of the object to be predicted.
  • the trajectory prediction model is a second trajectory prediction model, and the second trajectory prediction model is used to predict the motion trajectories of all moving objects in the area; in this case, the prediction module 64 specifically uses in:
  • the object to be predicted includes at least one of the following: vehicles, animals, humans, robots, and unmanned aerial vehicles.
  • the trajectory prediction device 600 may further include:
  • the planning module (not shown in FIG. 9) is used to perform motion planning for the object to be predicted according to the target trajectory.
  • the trajectory prediction device 600 of the embodiment shown in FIG. 9 can be used to implement the technical solutions of the above method embodiments.
  • the trajectory prediction device 600 can be Terminal equipment or background server, etc.
  • the division of the various modules of the trajectory prediction apparatus 600 shown in FIG. 9 is only a division of logical functions, and may be fully or partially integrated into a physical entity in actual implementation, or may be physically separated.
  • these modules can all be implemented in the form of software called by processing elements; they can also be implemented in the form of hardware; part of the modules can be implemented in the form of software called by the processing elements, and some of the modules can be implemented in the form of hardware.
  • the prediction module 64 may be a separate processing element, or it may be integrated in the trajectory prediction device 600, such as implemented in a certain chip of the terminal. In addition, it may also be stored in the memory of the trajectory prediction device 600 in the form of a program.
  • a certain processing element of the trajectory prediction device 600 calls and executes the functions of the above modules.
  • the implementation of other modules is similar.
  • all or part of these modules can be integrated together or implemented independently.
  • the processing element described here may be an integrated circuit with signal processing capability.
  • each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or instructions in the form of software.
  • the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital singnal processor, DSP), or, one or more field programmable gate arrays (Field Programmable Gate Array, FPGA), etc.
  • ASIC application specific integrated circuits
  • DSP digital singnal processor
  • FPGA Field Programmable Gate Array
  • the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call programs.
  • these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).
  • SOC system-on-a-chip
  • the trajectory prediction device 600 includes:
  • the computer program is stored in the memory 610 and is configured to be executed by the processor 620 to implement the method described in the foregoing embodiment.
  • the number of processors 620 in the trajectory prediction apparatus 600 may be one or more, and the processors 620 may also be referred to as processing units, which may implement certain control functions.
  • the processor 620 may be a general-purpose processor or a special-purpose processor.
  • the processor 620 may also store instructions, and the instructions may be executed by the processor 620 so that the trajectory prediction apparatus 600 executes the trajectory prediction method described in the foregoing method embodiment.
  • the trajectory prediction device 600 may include a circuit, which may implement the sending or receiving or communication function in the foregoing method embodiment.
  • the number of memories 610 in the trajectory prediction device 600 may be one or more, and instructions or intermediate data are stored on the memory 610, and the instructions may be executed on the processor 620, so that the trajectory The prediction device 600 executes the method described in the above method embodiment.
  • the memory 610 may also store other related data.
  • the processor 620 may also store instructions and/or data. The processor 620 and the memory 610 may be provided separately or integrated together.
  • the trajectory prediction device 600 is also provided with a transceiver 630, where the transceiver 630 may be called a transceiver unit, a transceiver, a transceiver circuit, or a transceiver, etc., for testing
  • the device or other terminal devices perform data transmission or communication, which will not be repeated here.
  • the memory 610, the processor 620, and the transceiver 630 are connected and communicate via a bus.
  • the transceiver 630 may obtain global semantic data and global trajectory data.
  • the processor 620 is configured to complete corresponding determination or control operations, and optionally, may also store corresponding instructions in the memory 610. For the specific processing manner of each component, reference may be made to the related description of the foregoing embodiment.
  • an embodiment of the present invention provides a readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method as described in the first embodiment.
  • the driving system 800 includes:
  • the trajectory prediction device 600 is configured to execute the method described in any implementation manner in the first embodiment
  • the motion controller 810 is configured to control the motion of the controlled object according to the target trajectory obtained by the trajectory prediction device.
  • the controlled object and the object to be predicted are the same object.
  • the motion controller in the vehicle's own trajectory prediction and route planning scenario, can plan its own driving route according to its own target trajectory predicted by the trajectory prediction device 600; and, further Therefore, automatic movement of the controlled object can be realized, that is, automatic driving can be realized.
  • the controlled object and the object to be predicted may be different objects.
  • the vehicle can predict the trajectory of other vehicles on the road that are closer to itself, so that when performing its own motion control, it can try to avoid other vehicles or moving obstacles (vehicles, people, animals, etc.) In order to reduce the probability of safety accidents, it is beneficial to improve safety.
  • the motion controller 810 may also output the target trajectory acquired by the trajectory prediction device 600, so that the user can use the target trajectory as a reference when driving or controlling the movement of the controlled object. Especially in a scene with multiple objects, by predicting the target trajectory of multiple other objects to be predicted, it is more conducive to improving the control security in the scene with multiple objects.
  • the controlled object may include but is not limited to at least one of the following: vehicles, animals, people, robots, and unmanned aerial vehicles.
  • the number of controlled objects in the embodiment of the present invention is not particularly limited, and may be one or more.
  • the motion controller may be a driving controller of a vehicle, or a flight controller of an unmanned aerial vehicle, etc., which will not be repeated.
  • an embodiment of the present invention provides a vehicle.
  • the vehicle 900 includes:
  • the trajectory prediction device 600 is configured to execute the method described in any implementation manner in the first embodiment.
  • the vehicle 900 includes:
  • the driving system 800 is shown in FIG. 11.
  • the embodiment of the present invention also provides a control device for an unmanned aerial vehicle.
  • control device of the UAV includes:
  • the trajectory prediction device 600 is configured to execute the method described in any implementation manner in the first embodiment.
  • control device of the unmanned aerial vehicle includes:
  • the unmanned aerial vehicle and the control device of the unmanned aerial vehicle can be designed independently or in combination (the control device is arranged inside the unmanned aerial vehicle), which is not particularly limited in the embodiment of the present invention.
  • control devices of vehicles and unmanned aerial vehicles are controlled objects that can carry the aforementioned trajectory prediction device, as mentioned above, in addition to this, they may further include robots or machine toys, etc., which will not be repeated.
  • a person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware.
  • the foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Traffic Control Systems (AREA)

Abstract

提供了一种轨迹预测方法及装置、存储介质、驾驶系统与车辆。该方法通过获取待预测对象所在区域的全局语义数据与全局轨迹数据（S202），然后，融合全局语义数据与全局轨迹数据，得到全局融合数据（S204），从而，提取全局融合数据中的特征，得到全局特征（S206），进而，利用训练好的轨迹预测模型处理全局特征，得到待预测对象的目标轨迹（S208）。该方法能够结合全局数据实现对运动对象的轨迹预测，具备较高的预测准确率，并在一定程度上降低意外事故的发生概率。

Description

轨迹预测方法及装置、存储介质、驾驶系统与车辆 技术领域
本发明涉及智能交通技术领域，尤其涉及一种轨迹预测方法及装置、存储介质、驾驶系统与车辆。
背景技术
随着智能交通领域的发展，对运动对象的运动轨迹的预测算法在路径规划领域具备重大意义。通过对运动对象的运动轨迹进行预测，能够在已知运动对象未来可能的运动轨迹的情况下进行路径规划，有利于防止碰撞等意外情况的发生。
目前的轨迹预测算法一般以运动对象自身的运动数据为基础，根据运动对象所属类别确定运动对象适用的运动模型，并利用该运动模型处理运动对象自身的运动数据，之后，再通过后处理的方式整合区域语义信息，即可预测出待预测对象的运动轨迹。
现有的轨迹预测算法以运动对象自身的运动数据为基础，无法从全局出发进行轨迹预测，这容易导致不同运动对象的预测轨迹出现交叉，进而导致以此为依据的路径规划或调度发生碰撞等意外事故，存在较大的安全隐患。
发明内容
本发明实施例提供一种轨迹预测方法及装置、存储介质、驾驶系统与车辆，能够结合全局数据实现对运动对象的轨迹预测，具备较高的预测准确率，并在一定程度上降低意外事故的发生概率。
第一方面，本发明实施例提供了一种轨迹预测方法，包括：
获取待预测对象所在区域的全局语义数据与全局轨迹数据；
融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据；
提取所述全局融合数据中的特征，得到全局特征；
利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
第二方面，本发明实施例提供了一种轨迹预测装置，包括：
获取模块，用于获取待预测对象所在区域的全局语义数据与全局轨迹数据；
融合模块，用于融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据；
特征提取模块，用于提取所述全局融合数据中的特征，得到全局特征；
预测模块，用于利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
第三方面，本发明实施例提供了一种轨迹预测装置，包括：
存储器；
处理器；以及
计算机程序；
其中，所述计算机程序存储在所述存储器中，并被配置为由所述处理器执行以实现如第一方面所述的方法。
第四方面，本发明实施例提供了一种计算机可读存储介质，其上存储有计算机程序，
所述计算机程序被处理器执行以实现如第一方面所述的方法。
第五方面，本发明实施例提供了一种驾驶系统，包括：
轨迹预测装置，用于执行如第一方面所述的方法；
运动控制器，用于根据所述目标轨迹控制被控制对象运动。
一种可能的设计中，所述被控制对象与所述待预测对象为不同对象。
第六方面，本发明实施例提供了一种车辆，包括：
如第二方面或第三方面所述的轨迹预测装置，用于执行如第一方面所述的方法。
第七方面，本发明实施例提供了一种车辆，包括：
如第五方面所述的驾驶系统。
第八方面，本发明实施例提供了一种无人飞行器的控制装置，包括：
如第五方面所述的驾驶系统。
本发明实施例所提供的技术方案，通过对待预测对象所在区域的全局语义数据与全局轨迹数据进行获取与处理，能够得到该区域的全局特征，进而， 利用训练好的轨迹预测模型对全局特征进行处理，即可得到待预测对象的目标轨迹，换言之，本发明实施例所提供的技术方案从全局语义和全局轨迹出发，在预测一个待预测对象的轨迹时，考虑该区域内的全部运动对象，并结合该区域的全局语义数据，以实现对该区域内任一运动对象的轨迹预测，相较于仅考虑单一的待预测对象的预测方法，本方案具备较高的预测准确率，并且，当以此为依据执行后续的路径规划或调度，也能够在一定程度上降低意外事故的发生概率，具备更高的安全性。
附图说明
图1为本发明实施例提供的一种轨迹预测场景的俯视示意图；
图2为本发明实施例提供的一种轨迹预测方法的流程示意图；
图3为本发明实施例提供的另一种轨迹预测方法的流程示意图；
图4为本发明实施例提供的另一种轨迹预测方法的流程示意图；
图5为本发明实施例提供的循环神经网络模型的循环单元结构示意图；
图6为本发明实施例提供的循环神经网络模型的模型架构示意图；
图7为本发明实施例提供的长短期记忆网络模型的模型架构示意图；
图8为本发明实施例提供的另一种轨迹预测方法的流程示意图；
图9为本发明实施例提供的一种轨迹预测装置的功能方块图；
图10为本发明实施例提供的一种轨迹预测装置的实体结构示意图；
图11为本方面实施例提供的一种驾驶系统的架构示意图；
图12为本发明实施例提供的一种车辆的架构示意图；
图13为本发明实施例提供的另一种车辆的架构示意图。
通过上述附图，已示出本公开明确的实施例，后文中将有更详细的描述。这些附图和文字描述并不是为了通过任何方式限制本公开构思的范围，而是通过参考特定实施例为本领域技术人员说明本公开的概念。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一 致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
首先对本发明所涉及的名词进行解释：
运动对象：是指能够实现轨迹移动的生物或物体。本发明实施例所涉及到的运动对象可以包括但不限于：车辆、动物、人、机器人与无人飞行器中的至少一种。其中，车辆可以为无人驾驶车辆，如无人地面车辆(Unmanned Ground Vehicle，UGV)，或者，处于自动驾驶模式的私家车辆或公交车辆等。
待预测对象：待预测目标轨迹的一个或多个运动对象。其中，多个是指两个或两个以上，后续涉及该概念，不再赘述。
语义对象：是指区域中各个语义概念上的对象。请参考图1所示场景，该场景中的语义对象包括：车辆、车道线、车道。可知，图1仅用以示例，在实际的轨迹预测场景中，语义对象的语义类别还包括多种，如：树木、障碍物、栏杆、指示牌、人、动物等，本发明实施例对于各语义对象的语义类别无特殊限定。
长短期记忆网络(Long Short Term Memory，LSTM)模型：是循环神经网络(Recurrent Neural Network，RNN)的一种变种模型，相较于RNN模型，LSTM具备更长的时间依赖建模能力。
本发明实施例所提供的技术方案具体的应用场景为：针对运动对象的轨迹预测场景。
进一步的，本发明实施例所提供的技术方案还可以具体应用于路径规划场景，此时，可根据预测出的轨迹，来实现对某一个或多个运动对象的路径规划。
此外，本发明实施例所提供的技术方案还可以具体应用于车辆调度场景。例如，通过对其他不可调度车辆或对象的轨迹预测，来实现对可调度车辆的调度。
如背景技术所述，现有的轨迹预测方法仅是针对单独的一个运动对象，当确定该运动对象的类别后，则通过该类别对应的运动模型，对该运动对象自身的运动数据进行处理，从而，预测出该运动对象的运动轨迹。一方面，这种预测方式受制于对象类别的限制，需要准确判断出运动对象的类别，才能够以该类别对应的运动模型得到较为准确的预测结果；另一方面，这种预 测方式仅依赖于运动对象本身的运动数据，并未从全局角度结合当前运动环境、其他运动对象的运动情况进行综合分析，这种未考虑运动对象所在区域内的其他运动或非运动的对象，极有可能预测出的两个同一类别的运动对象的轨迹相交，从而，若以此为依据进行路径规划或对象调度，极有可能发生碰撞等意外事故，存在较大的安全性风险。
基于此，本发明实施例提供的技术方案，旨在解决现有技术的如上技术问题，并提出如下解决思路：综合考虑待预测对象所在区域的全局数据，包括全局语义数据与全局轨迹数据，据此得到全局特征，并以全局特征作为轨迹预测模型的输入，来获取待预测对象的目标轨迹。
基于这种设计，本发明实施例所提供的轨迹预测方法可以具体执行于某一运动对象的内置处理器或所持的终端设备中，或者，也可以具体执行于云端或后台服务器。
举例说明。在一种可能的场景中，自动驾驶车辆的第一处理器可自行规划行驶路线，而该自动驾驶车辆的第二处理器用于执行本方案所提供的轨迹预测方法，并用于将预测到的轨迹输入第一内置理器，以便于第一处理器可根据预测出的轨迹进行后续的路径规划。其中，第一处理器和第二处理器可以为同一处理器，或者，也可以为不同处理器，例如，可以为高级驾驶辅助系统(Advanced Driving Assistant System，ADAS)中的一个或两个处理器；以及，第一处理器和第二处理器可以为车辆的总控制器中的一部分，或者，也可以为控制该无人驾驶车辆行驶的后台总服务器或云端服务器。
下面以具体地实施例对本发明的技术方案以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本发明的实施例进行描述。
实施例一
本发明实施例提供了一种轨迹预测方法。请参考图2-图4，其中，图2示出了本发明实施例提供的一种轨迹预测方法的流程示意图，图3示出了本发明式实施例提供的轨迹预测方法在一种具体的应用场景下的实现流程示意图，图4为图3所示流程的一种具体实现方式。
如图2所示，该方法包括如下步骤：
S202，获取待预测对象所在区域的全局语义数据与全局轨迹数据。
如前所述，待预测对象为一种运动对象，其可以包括但不限于如下至少一种：车辆、动物、人、机器人与无人飞行器。此外，本发明实施例中对待预测对象的数目无特别限定，可以为一个或多个。
其中，全局语义数据用于描述待预测对象所在区域中各对象的语义类别，此时，各运动对象或非运动对象均作为语义对象，具备各自的语义类别。全局轨迹数据用于描述该区域中各运动对象的历史运动坐标。可知，在具体实现时，全局轨迹数据可以包括至少一帧的历史运动坐标。
图3示出了以图1所示场景为例时，本方案的实现流程示意图。如图3所示，该行驶场景中包含3个语义，分别是：车辆、车道与车道线时，此时，需要在该场景中获取的全局语义数据，也就是各语义对象的语义数据；以及，还需要获取该场景的全局轨迹数据，也就是各运动对象：车辆1(作为待预测对象)、车辆2和车辆3的轨迹数据。
需要说明的是，图3仅为示意性的，在具体的实现场景中，全局轨迹数据与全局语义数据的表现形式不局限于单个获取的方式，可作为整体数据被获取到。
在如图4所示的一种实现场景中，各运动对象的轨迹数据可以通过LSTM模型来实现，后续具体说明。
S204，融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据。
具体的，如图3所示，融合模块用于将每一帧对应的全局语义数据与全局轨迹数据进行融合，得到各帧的全局融合数据。
S206，提取所述全局融合数据中的特征，得到全局特征。
如图3所示，特征提取模块用于在全局融合数据中提取出全局特征。在如图4所示的一种实现场景中，该步骤可以通过卷积神经网络(Convolutional Neural Networks，CNN)模型来实现，后续具体说明。
S208，利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
本发明采用训练好的轨迹预测模型来预测运动对象的运动轨迹，此时，该轨迹预测模型的输入为全局特征，输出为运动对象的运动轨迹。
具体而言，本发明实施例所提供的轨迹预测模型可以包括但不限于：如 下至少一种：LSTM模型、多层感知器(Multi-Layer Perception，MLP)；其中，所述多层感知器包括：RNN模型、门控循环单元(gated recurrent neural network，GRU)模型。例如，图4所示的实现场景中，即通过LSTM模型来实现轨迹预测，此时，Y＝LSTM(feature)，其中，Y表示目标轨迹，feature表示全局特征。
请参考图5-7，其中，图5示出了RNN模型中一个循环单元的设计逻辑，图6示出了RNN模型的模型架构示意图，图7示出了LSTM模型的模型架构示意图。
RNN模型作为一种时序建模的有效手段，相比于普通的神经网络，如图5所示，其区别主要在于将上一帧的输出或者中间状态作为当前帧的输入，以实现对历史消息及时序关系的融合。将如图5所示的循环单元在时间展开后，即可得到如图6所示的RNN模型。如图6所示，RNN模型可以实现时序建模。因此，可以通过RNN模型来实现本方案中的轨迹预测步骤。
如图7所示，在LSTM模型中的每一个重复模块中均包含4个交互层，这4个交互层以特殊的方式进行交互，使得上一帧的输出或者中间状态作为当前帧的输入，从而，相比于图6所示的RNN模型，LSTM模型具备更优异的时间依赖建模能力。此外，考虑到针对轨迹的预测与运动对象的历史数据具备较强的时间关联关系，因此，采用LSTM模型来实现轨迹预测可得到更为接近实际发展的轨迹预测结果。
通过前述设计，本发明实施例所提供的技术方案能够从全局数据出发，实现对任一待预测对象的轨迹预测，相较于仅以待预测对象自身的运动数据出发的轨迹预测方式，本方案具备较高的预测准确率，并且，当以此为依据执行后续的路径规划或调度，也能够在一定程度上降低意外事故的发生概率，具备更高的安全性。
以下，对图2所示方法的实现方式进行具体说明。
S202包含两方面全局数据的获取：全局语义数据与全局轨迹数据。这两种全局数据的获取方式可以参考图8所示流程。
一方面，如图8所示，获取全局语义数据的方式可以包括如下步骤：
S202-12，获取所述待预测对象所在区域的全局区域图像。
该全局区域图像可以为实时获取到的图像，实时获取到的图像具备更高 的及时性，以此得到的全局语义数据也更为准确。具体而言，实时获取方式可以包括但不限于：通过图像采集设备实时采集图像。其中，图像采集设备可以为该轨迹预测装置(轨迹预测方法的执行主体)的一部分；或者，与该轨迹预测装置具备实时地数据交互。举例说明，若该轨迹预测装置可以为车辆的主控制器中的处理器A，则图像采集设备可以为车辆黑匣子中的摄像头，其可以将采集到的图像直接输入处理器A；或者，图像采集设备可以为区域内设置的摄像头，如道路上或路边设置的摄像头，此时，处理器A可以向该区域内摄像头或该区域内摄像头的后台服务器实时请求并接收全局区域图像。
除实时获取的方式之外，还可以通过调用已采集数据的方式来获取全局语义数据。具体的，一种实现场景中，可以获取高精度地图中关于该区域的全局区域图像；另一种实现场景中，可以获取其他处理器或存储器中已采集到的该区域的全局区域图像。并且，这种实现方式只能获取到该区域的环境信息，如道路、指示牌、车道线等非运动对象的非实时数据，而无法获取到实时场景中的运动对象的运动情况，因此，在以此方式实现本方案时，仅适用于针对单一待预测对象的轨迹预测，而无法结合其他运动对象的轨迹实现综合预测，预测准确率较弱。
需要说明的是，本发明实施例中后续进行处理的图像为俯视图像，因此，若以前述实现方式获取到的并非俯视图像时，还需要对采集到的图像进行俯视投影，以得到满足后续处理需要的俯视图像。
一种可能的设计中，本发明实施例所涉及到的俯视图像可以具体表现为：数字正射影像(Digital Orthophoto Map，DOM)图像。
此外，为了便于处理，还可以进一步个性化设置“待预测对象所在区域”的区域形状或尺寸，本发明实施例对此无特别限定。具体的，可以以待预测对象为中心，获取俯视图一定长宽尺寸的矩形区域为其所在区域，例如，可以获取如图1(或图3)所示的长为W、宽为H的矩形区域的图像。又例如，还可以将待预测对象所在的整条道路作为待预测对象所在区域。
S202-14，对所述全局区域图像中的各像素分别进行语义识别，得到各像素的语义类别。
一种可能的设计中，可以通过深度学习来实现对各像素的语义识别。若通过该方式实现，需要在执行该步骤之前，利用预设的像素样本数据，对像 素语义识别模型进行深度学习，以得到满足应用需求(可通过对损失函数的定义实现)的像素语义识别模型。如此，在执行该步骤时，仅需将全局区域图像输入该像素语义识别模型，该像素语义识别模型的输出即为各像素的语义类别。
另一种可能的设计中，还可以各像素的像素值为依据，将各像素的像素值分别与各语义类别对应的像素区间进行比较，从而，针对任一像素值，将该像素值落入的像素区间对应的语义类别，作为该像素对应的语义类别。其中，各语义类别与像素区间之间的对应关系，可以通过自定义方式预设。
S202-16，根据各像素的语义类别，对所述全局区域图像进行语义标注，得到所述全局语义信息。
由于各像素的语义类别已经确定，在执行该步骤时，可根据各像素的语义类别，对所述全局区域图像进行语义分割，得到多个语义对象；从而，对对各语义对象分别进行语义标注，得到所述全局语义信息。
其中，语义标注仅用于区分各对象的语义类别，可以任意可区分方式来进行标注。例如，可以通过不同的颜色来标识各语义对象。或者，如图1(或图3)所示，可以通过不同的底纹来标识各语义对象。可知，经过标识后，具备同样标识的语义对象为同一类语义对象。
此时，需要说明的是，在不同的实现场景中，与该待预测对象所属类别相同的其他运动对象的标注方式，可以与待预测对象的标注方式相同，也可以不同。如图1所示，当待预测对象为车辆1时，该待预测对象所在区域还包括同一类别的运动对象：车辆2与车辆3。此时，若通过第一轨迹预测模型(第一轨迹预测模型用于预测所述待预测对象的目标轨迹，后续详述实现方式)来实现S208步骤时，如图1所示，需要将车辆1与车辆2、车辆3进行区分标识，车辆1具备一种标识，车辆2与车辆3为另一种标识。或者，若通过第二轨迹预测模型(第二轨迹预测模型用于预测所述区域中全部运动对象的运动轨迹，后续详述实现方式)来实现S208步骤时，则无需对同一类别的运动对象进行区分标识，车辆1、车辆2与车辆3可使用同一种标识方式进行标识(同种标识的方式图1未示出)。
以及，在执行语义标注时，还可以进一步自定义划分网格，每个网格可以包括一个或多个像素点，其划分方式可以通过预设的分辨率来实现。例如， 可以将图1所示的全局区域图像划分为长度为20cm的正方形网格，如此，在执行后续的语义标注时，仅需要对网格进行标注即可。当网格包含多个像素点时，划分网格的实现方式有利于降低标记量，提高了处理效率。
除前述实现方式之外，图8所述的S202-14与S202-16步骤还可以通过一个神经网络模型来实现。也就是，在执行S202-14之前，训练语义识别模型，如此，将S202-12步骤获取到的全局区域图像输入该语义识别模型，该语义识别模型的输出即为标注了语义类别的全局区域图像，也就得到了全局语义数据。
针对前述涉及到的语义识别模型、像素语义识别模型的类别无特别限定，采用CNN模型或其他神经网络模型均可实现，而二者的样本数据则需要根据模型的输入和输出做不同的标注何设计，不再赘述。
另一方面，如图8所示，获取全局轨迹数据的方式可以包括如下步骤：
S202-22，获取所述待预测对象所在区域内各运动对象的轨迹点集，所述轨迹点集由所述运动对象的坐标点按照时序顺序集合而成。
该步骤用于获取当前区域中的各运动对象的轨迹点集，其中，每个轨迹点集由该运动对象的多个坐标点构成，为了便于处理，可以将各运动对象的坐标点转换为同一个坐标系下的坐标点。例如，可以将各轨迹点集中的坐标点转换为以图1所示的矩形区域的两条直角边构成的直角坐标系中的坐标点，每个坐标点的表现形式为(X，Y)，而每个运动对象的轨迹点集可表示为{(X i，Y i)}，其中i用于表示个坐标点的时序顺序。
具体而言，在具体实现时，可通过获取每个运动对象在以当前时刻为终点的时间区间内的坐标点构成前述坐标点集。其中，时间区间的长度可根据需要预设，例如，可以获取当前时刻前3s内各运动对象的轨迹点集。
需要说明的是，该步骤中，轨迹点集可以是该执行主体主动监测到的，也可以是通过向其他处理器或采集装置请求数据得到的。例如，若执行主体为车辆1的主控制器中的处理器A，则本车辆1的坐标点可以通过自身的定位器，如GPS，采集得到，由本车辆1的定位器将采集到的坐标数据发送给处理器A，由处理器A进行坐标转换，得到本车辆1的轨迹点集；而其他车辆的轨迹点集则可以通过向其他处理器请求的方式得到，例如，若与其他车辆存在通信，可分别向其他车辆获取其轨迹点集；又例如，可以向该区域的 路面监视器获取其他车辆的轨迹点集；此外，也可以通过自身采集其他车辆的图像并计算与自身的间距的方式，计算获取到其他车辆的轨迹点集。实现方式可以有多种，不再赘述。
此外，该轨迹点集的数据源(或直接采集源)与前述S202-12中全局区域图像的数据源(或直接采集源)可以不同。
S202-24，对各运动对象的所述轨迹点集进行编码处理，得到各运动对象的轨迹特征。
该步骤也可以通过神经网络模型来实现，将前述S202-22得到的各运动对象的轨迹点集(例如，各运动对象在3s内的轨迹点集)输入编码模型(一种训练好的神经网络模型，例如，可采用如图4所示的LSTM模型)，该编码模型的输出即为各运动对象的轨迹特征(encoder)。
具体的，针对任一运动对象而言，其轨迹特征可以表现为：encoder＝LSTM{(X i，Y i)}。其中，轨迹特征(encoder)的长度可以假设为C，C的取值一般为预设经验值。
S202-26，根据各运动对象的所述轨迹特征，构建轨迹张量，以作为所述全局轨迹数据。
基于S202-24步骤中获取到的各运动对象的轨迹特征，该步骤可以构建一个轨迹张量(tensor)，其尺寸为C*H*W，将各运动对象的轨迹特征(encoder)对应存放至该tensor中即可。具体的，针对任一运动对象，将该运动对象的encoder对应存储在该运动对象的中心位置。一种可能的设计中，也可按照图1所示方式，在tensor中划分网格，如此，该运动对象的中心位置位于该Tensor中的哪一个网格，就将该运动对象的encoder对应存储在该网格中即可。
通过如图8所示的实现方式，可以实现对全局语义数据与全局轨迹数据的获取。如前所述的实现方式中，全局语义数据可以表现为一个W*H的图像，而全局轨迹数据则表现为一个尺寸为C*H*W的tensor，因此，在执行S204所述的融合步骤时，可将二者融合为一个尺寸为(C+1)*H*W的融合tensor。该融合tensor可具体表示为：tensor((C+1)*H*W)。
基于前述步骤得到的全局融合数据，只需要再对该全局融合数据进行特征提取，即可得到包含待预测对象在内的全局特征。具体实现时，也可以通过神经网络模型实现。也就是，利用训练好的特征提取模型处理所述融合信 息，得到所述全局特征。其中，本发明实施例所涉及到的特征提取模型至少包括：如图4所示的卷积神经网络(Convolutional Neural Networks，CNN)模型。与前述利用神经网络模型实现数据处理的方式类似，需要在执行该步骤之前，利用特征提取样本对该CNN模型进行训练。模型训练过程不再赘述。
同样的，在执行S208之前，也需要完成针对轨迹预测模型的训练学习。在具体的实现场景中，一般在执行本方案之前即完成轨迹预测模型(以及前述各实现方式中所涉及到的神经网络模型)的训练，以便于实时高效地实现轨迹预测，这种实现方式具备较高的实时性，有利于在实时实现轨迹预测，进而近似实时地实现路径规划或调度。
在具体实现轨迹预测模型的训练时，可以训练出根据全局特征预测某一单一运动对象(待预测对象)的第一轨迹预测模型。针对待预测对象而言，这种单一预测方式具备更快的处理效率，有利于实时场景下的路径规划和调度。
或者，也可以训练出根据全局特征预测区域内包含的全部运动对象的第二轨迹预测模型。其中，当利用第二轨迹预测模型处理全局特征时，将所述全局特征输入所述第二轨迹预测模型，获取所述第二轨迹预测模型输出的所述区域中全部运动对象的运动轨迹，并将所述全部运动对象中的所述待预测对象的运动轨迹，作为所述目标轨迹即可。这种全局预测方式能够一次性输出区域内全部运动对象的运动轨迹，有利于全局调度的实现，也有利于降低调度或路径规划过程中意外事故的发生概率，提高安全性。
此外，还可以训练出根据全局特征预测区域内包含的多个(非全部)运动对象的第三轨迹预测模型，模型训练方式及S208的实现方式同上，不再赘述。
综上，基于所训练出的轨迹预测模型的设计不同，本发明实施例所提供的技术方案不仅能实现针对单一运动对象的轨迹预测，还能够实现多个运动对象的轨迹预测。且如上所述的轨迹预测模型对对象类别无依赖，通过上述训练好的轨迹预测模型，可实现对任意类别的运动对象的轨迹预测，具备更高的灵活性，还可适用于具备多类对象的场景中的轨迹预测。
此外，在一些特殊的实现场景中，也可以如现有实现方式，分别为各类运动对象分别训练各自对应的轨迹预测模型。也就是，本发明实施例所提供 的技术方案也可针对各类别的运动对象实现个性化预测。其中，为各类运动对象分别训练各自的轨迹预测模型时，样本数据为该类别对象的相关数据。
如前所述，通过前述各实现方式得到的待预测对象的目标轨迹之后，即可利用该目标轨迹作进一步处理。
一种可能的设计中，可以根据所述目标轨迹，为所述待预测对象进行运动规划。也就是，根据该待预测对象的预测轨迹来实现进一步的路径规划。
另一种可能的设计中，可以根据所述目标轨迹，为其他运动对象进行运动规划。也就是，在为其他某一个或多个运动对象进行路径规划的过程中，可以根据该待预测对象的预测轨迹来规划路线，以避免和该待预测对象发生碰撞或其他安全事故。
进而，根据规划好的运动路径，实现运动对象的调度。
可以理解的是，上述实施例中的部分或全部步骤或操作仅是示例，本申请实施例还可以执行其它操作或者各种操作的变形。此外，各个步骤可以按照上述实施例呈现的不同的顺序来执行，并且有可能并非要执行上述实施例中的全部操作。
实施例二
基于上述实施例一所提供的轨迹预测方法，本发明实施例进一步给出实现上述方法实施例中各步骤及方法的装置实施例。
本发明实施例提供了一种轨迹预测装置，请参考图9，该轨迹预测装置600，包括：
获取模块61，用于获取待预测对象所在区域的全局语义数据与全局轨迹数据；
融合模块62，用于融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据；
特征提取模块63，用于提取所述全局融合数据中的特征，得到全局特征；
预测模块64，用于利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
一种可能的设计中，所述获取模块61，具体用于：
获取所述待预测对象所在区域的全局区域图像；
对所述全局区域图像中的各像素分别进行语义识别，得到各像素的语义类别；
根据各像素的语义类别，对所述全局区域图像进行语义标注，得到所述全局语义信息。
其中，所述获取模块61，还进一步具体用于：
根据各像素的语义类别，对所述全局区域图像进行语义分割，得到多个语义对象；
对各语义对象分别进行语义标注，得到所述全局语义信息。
本发明实施例所涉及到的所述全局区域图像为数字正射影像DOM图像。
另一种可能的设计中，所述获取模块61，具体用于：
获取所述待预测对象所在区域内各运动对象的轨迹点集，所述轨迹点集由所述运动对象的坐标点按照时序顺序集合而成；
对各运动对象的所述轨迹点集进行编码处理，得到各运动对象的轨迹特征；
根据各运动对象的所述轨迹特征，构建轨迹张量，以作为所述全局轨迹数据。
一种可能的设计中，所述融合模块63，具体用于：
利用训练好的特征提取模型处理所述融合信息，得到所述全局特征。
其中，本发明实施例所涉及到的所述特征提取模型至少包括：卷积神经网络CNN模型。
本发明实施例所涉及到的所述轨迹预测模型包括如下至少一种：长短期记忆网络LSTM模型、多层感知器MLP；
其中，所述多层感知器包括：循环神经网络RNN模型、门控循环单元GRU模型。
一种可能的设计中，所述轨迹预测模型为第一轨迹预测模型，所述第一轨迹预测模型用于预测所述待预测对象的目标轨迹。
另一种可能的设计中，所述轨迹预测模型为第二轨迹预测模型，所述第二轨迹预测模型用于预测所述区域中全部运动对象的运动轨迹；此时，预测模块64，具体用于：
将所述全局特征输入所述第二轨迹预测模型，获取所述第二轨迹预测模 型输出的所述区域中全部运动对象的运动轨迹，并将所述全部运动对象中的所述待预测对象的运动轨迹，作为所述目标轨迹。
本发明实施例中，所述待预测对象包括如下至少一种：车辆、动物、人、机器人与无人飞行器。
此外，一种或可能的设计中，轨迹预测装置600还可以包括：
规划模块(图9未示出)，用于根据所述目标轨迹，为所述待预测对象进行运动规划。
图9所示实施例的轨迹预测装置600可用于执行上述方法实施例的技术方案，其实现原理和技术效果可以进一步参考方法实施例中的相关描述，可选的，该轨迹预测装置600可以是终端设备或后台服务器等。
应理解以上图9所示轨迹预测装置600的各个模块的划分仅仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。且这些模块可以全部以软件通过处理元件调用的形式实现；也可以全部以硬件的形式实现；还可以部分模块以软件通过处理元件调用的形式实现，部分模块通过硬件的形式实现。例如，预测模块64可以为单独设立的处理元件，也可以集成在轨迹预测装置600中，例如终端的某一个芯片中实现，此外，也可以以程序的形式存储于轨迹预测装置600的存储器中，由轨迹预测装置600的某一个处理元件调用并执行以上各个模块的功能。其它模块的实现与之类似。此外这些模块全部或部分可以集成在一起，也可以独立实现。这里所述的处理元件可以是一种集成电路，具有信号的处理能力。在实现过程中，上述方法的各步骤或以上各个模块可以通过处理器元件中的硬件的集成逻辑电路或者软件形式的指令完成。
例如，以上这些模块可以是被配置成实施以上方法的一个或多个集成电路，例如：一个或多个特定集成电路(Application Specific Integrated Circuit，ASIC)，或，一个或多个微处理器(digital singnal processor，DSP)，或，一个或者多个现场可编程门阵列(Field Programmable Gate Array，FPGA)等。再如，当以上某个模块通过处理元件调度程序的形式实现时，该处理元件可以是通用处理器，例如中央处理器(Central Processing Unit，CPU)或其它可以调用程序的处理器。再如，这些模块可以集成在一起，以片上系统(system-on-a-chip，SOC)的形式实现。
并且，本发明实施例提供了一种轨迹预测装置，请参考图10，该轨迹预测装置600，包括：
存储器610；
处理器620；以及
计算机程序；
其中，计算机程序存储在存储器610中，并被配置为由处理器620执行以实现如上述实施例所述的方法。
其中，轨迹预测装置600中处理器620的数目可以为一个或多个，处理器620也可以称为处理单元，可以实现一定的控制功能。所述处理器620可以是通用处理器或者专用处理器等。在一种可选地设计中，处理器620也可以存有指令，所述指令可以被所述处理器620运行，使得所述轨迹预测装置600执行上述方法实施例中描述的轨迹预测方法。
在又一种可能的设计中，轨迹预测装置600可以包括电路，所述电路可以实现前述方法实施例中发送或接收或者通信的功能。
可选地，所述轨迹预测装置600中存储器610的数目可以为一个或多个，存储器610上存有指令或者中间数据，所述指令可在所述处理器620上被运行，使得所述轨迹预测装置600执行上述方法实施例中描述的方法。可选地，所述存储器610中还可以存储有其他相关数据。可选地处理器620中也可以存储指令和/或数据。所述处理器620和存储器610可以单独设置，也可以集成在一起。
此外，如图10所示，在该轨迹预测装置600中还设置有收发器630，其中，所述收发器630可以称为收发单元、收发机、收发电路、或者收发器等，用于与测试设备或其他终端设备进行数据传输或通信，在此不再赘述。
如图10所示，存储器610、处理器620与收发器630通过总线连接并通信。
若该轨迹预测装置600用于实现对应于图2中的方法时，例如，可以由收发器630获取全局语义数据与全局轨迹数据。而处理器620用于完成相应的确定或者控制操作，可选的，还可以在存储器610中存储相应的指令。各个部件的具体的处理方式可以参考前述实施例的相关描述。
此外，本发明实施例提供了一种可读存储介质，其上存储有计算机程序， 该计算机程序被处理器执行以实现如实施例一所述的方法。
此外，本发明实施例提供了一种驾驶系统，请参考图11，该驾驶系统800包括：
轨迹预测装置600，用于执行如实施例一中任一实现方式所述的方法；
运动控制器810，用于根据所述轨迹预测装置获取到的目标轨迹控制被控制对象运动。
其中，被控制对象与待预测对象为同一对象。例如，一种可能的设计中，车辆对自身的轨迹预测及路线规划场景中，运动控制器可根据轨迹预测装置600预测出的自身的目标轨迹，来实现对自身行驶路线的规划；以及，进一步地，可实现对被控制对象的自动运动，也就是，实现自动驾驶。
此外，被控制对象与待预测对象可以为不同对象。以前述场景为例，车辆可以对路面上与自身较为接近的其他车辆进行轨迹预测，以便于在执行自身的运动控制时，能够尽量避免与其他车辆或运动障碍物(车辆、人、动物等)的碰撞，以降低安全事故的发生概率，有利于提高安全性。
在另一具体的实现场景中，运动控制器810亦可将轨迹预测装置600获取到的目标轨迹进行输出，以便于用户在驾驶或控制被控制对象运动时，可以将该目标轨迹作为参考。尤其是在多个对象的场景中，通过对其他多个待预测对象进行目标轨迹的预测，更有利于提高多对象场景中的控制安全。
具体而言，被控制对象可以包括但不限于如下至少一种：车辆、动物、人、机器人与无人飞行器。此外，本发明实施例中对被控制对象的数目无特别限定，可以为一个或多个。例如，运动控制器可以为车辆的行驶控制器，也可以为无人飞行器的飞行控制器等，不再赘述。
此外，本发明实施例提供了一种车辆。
请参考图12，该车辆900包括：
轨迹预测装置600，用于执行如实施例一中任一实现方式所述的方法。
或者，如图13所示，该车辆900包括：
如图11所示的驾驶系统800。
此外，本发明实施例还提供了一种无人飞行器的控制装置。
一种可能的设计中，该无人飞行器的控制装置，包括：
轨迹预测装置600，用于执行如实施例一中任一实现方式所述的方法。
另一设计中，该无人飞行器的控制装置，包括：
驾驶系统800。
具体的，无人飞行器与无人飞行器的控制装置可以独立设计，也可以组合设计(该控制装置设置于无人飞行器内部)，本发明实施例对此无特别限定。
可知，车辆与无人飞行器的控制装置为可承载前述轨迹预测装置的被控制对象，如前说书，除此之外，还进一步包括机器人或机器玩具等，不再赘述。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
由于本实施例中的各模块能够执行实施例一所示的方法，本实施例未详细描述的部分，可参考对实施例一的相关说明。
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。

Claims (17)

  1. 一种轨迹预测方法，其特征在于，包括：
    获取待预测对象所在区域的全局语义数据与全局轨迹数据；
    融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据；
    提取所述全局融合数据中的特征，得到全局特征；
    利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
  2. 根据权利要求1所述的方法，其特征在于，所述获取待预测对象所在区域的全局语义数据，包括：
    获取所述待预测对象所在区域的全局区域图像；
    对所述全局区域图像中的各像素分别进行语义识别，得到各像素的语义类别；
    根据各像素的语义类别，对所述全局区域图像进行语义标注，得到所述全局语义信息。
  3. 根据权利要求2所述的方法，其特征在于，所述根据各像素的语义类别，对所述全局区域图像进行语义标注，得到所述全局语义信息，包括：
    根据各像素的语义类别，对所述全局区域图像进行语义分割，得到多个语义对象；
    对各语义对象分别进行语义标注，得到所述全局语义信息。
  4. 根据权利要求2或3所述的方法，其特征在于，所述全局区域图像为数字正射影像DOM图像。
  5. 根据权利要求1所述的方法，其特征在于，所述获取待预测对象所在区域的全局轨迹数据，包括：
    获取所述待预测对象所在区域内各运动对象的轨迹点集，所述轨迹点集由所述运动对象的坐标点按照时序顺序集合而成；
    对各运动对象的所述轨迹点集进行编码处理，得到各运动对象的轨迹特征；
    根据各运动对象的所述轨迹特征，构建轨迹张量，以作为所述全局轨迹数据。
  6. 根据权利要求1-3、5中任一项所述的方法，其特征在于，所述提取 所述全局融合数据中的特征，得到全局特征，包括：
    利用训练好的特征提取模型处理所述融合信息，得到所述全局特征。
  7. 根据权利要求6所述的方法，其特征在于，所述特征提取模型至少包括：卷积神经网络CNN模型。
  8. 根据权利要求1所述的方法，其特征在于，所述轨迹预测模型包括如下至少一种：长短期记忆网络LSTM模型、多层感知器MLP；
    其中，所述多层感知器包括：循环神经网络RNN模型、门控循环单元GRU模型。
  9. 根据权利要求1或8所述的方法，其特征在于，所述轨迹预测模型为第一轨迹预测模型，所述第一轨迹预测模型用于预测所述待预测对象的目标轨迹。
  10. 根据权利要求1或8所述的方法，其特征在于，所述轨迹预测模型为第二轨迹预测模型，所述第二轨迹预测模型用于预测所述区域中全部运动对象的运动轨迹；
    所述利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹，包括：
    将所述全局特征输入所述第二轨迹预测模型，获取所述第二轨迹预测模型输出的所述区域中全部运动对象的运动轨迹，并将所述全部运动对象中的所述待预测对象的运动轨迹，作为所述目标轨迹。
  11. 根据权利要求1所述的方法，其特征在于，所述待预测对象包括如下至少一种：车辆、动物、人、机器人与无人飞行器。
  12. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    根据所述目标轨迹，为所述待预测对象进行运动规划。
  13. 一种轨迹预测装置，其特征在于，包括：
    获取模块，用于获取待预测对象所在区域的全局语义数据与全局轨迹数据；
    融合模块，用于融合所述全局语义数据与所述全局轨迹数据，得到全局融合数据；
    特征提取模块，用于提取所述全局融合数据中的特征，得到全局特征；
    预测模块，用于利用训练好的轨迹预测模型处理所述全局特征，得到所述待预测对象的目标轨迹。
  14. 一种轨迹预测装置，其特征在于，包括：
    存储器；
    处理器；以及
    计算机程序；
    其中，所述计算机程序存储在所述存储器中，并被配置为由所述处理器执行以实现如权利要求1至12任一项所述的方法。
  15. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序，
    所述计算机程序被处理器执行以实现如权利要求1至12任一项所述的方法。
  16. 一种驾驶系统，其特征在于，包括：
    轨迹预测装置，用于执行如权利要求1至12任一项所述的方法；
    运动控制器，用于根据所述轨迹预测装置获取到的目标轨迹控制被控制对象运动。
  17. 一种车辆，其特征在于，包括：
    如权利要求16所述的驾驶系统。
PCT/CN2019/079780 2019-03-27 2019-03-27 轨迹预测方法及装置、存储介质、驾驶系统与车辆 WO2020191642A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/079780 WO2020191642A1 (zh) 2019-03-27 2019-03-27 轨迹预测方法及装置、存储介质、驾驶系统与车辆
CN201980005403.6A CN111316286A (zh) 2019-03-27 2019-03-27 轨迹预测方法及装置、存储介质、驾驶系统与车辆

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/079780 WO2020191642A1 (zh) 2019-03-27 2019-03-27 轨迹预测方法及装置、存储介质、驾驶系统与车辆

Publications (1)

Publication Number Publication Date
WO2020191642A1 true WO2020191642A1 (zh) 2020-10-01

Family

ID=71161143

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079780 WO2020191642A1 (zh) 2019-03-27 2019-03-27 轨迹预测方法及装置、存储介质、驾驶系统与车辆

Country Status (2)

Country Link
CN (1) CN111316286A (zh)
WO (1) WO2020191642A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112699942A (zh) * 2020-12-30 2021-04-23 东软睿驰汽车技术（沈阳）有限公司 一种运营车辆识别方法、装置、设备及存储介质
CN112712013A (zh) * 2020-12-29 2021-04-27 杭州海康威视数字技术股份有限公司 一种移动轨迹构建方法及装置
CN112885079A (zh) * 2021-01-11 2021-06-01 成都语动未来科技有限公司 一种基于全局注意力和状态共享的车辆轨迹预测方法
CN113011323A (zh) * 2021-03-18 2021-06-22 北京百度网讯科技有限公司 通行状态的获取方法、相关装置、路侧设备及云控平台
CN113139696A (zh) * 2021-05-11 2021-07-20 深圳大学 一种轨迹预测模型构建方法及轨迹预测方法、装置
CN113276858A (zh) * 2021-05-13 2021-08-20 际络科技(上海)有限公司 一种节油驾驶控制方法及装置、计算设备和存储介质
CN113743767A (zh) * 2021-08-27 2021-12-03 广东工业大学 基于时间和安全性的车辆派单方法、系统、计算机及介质
CN113759400A (zh) * 2021-08-04 2021-12-07 江苏怀业信息技术股份有限公司 卫星定位轨迹的平滑化方法和装置
CN113934808A (zh) * 2021-10-22 2022-01-14 广东汇天航空航天科技有限公司 地图数据获取方法、装置及飞行器
CN114512052A (zh) * 2021-12-31 2022-05-17 武汉中海庭数据技术有限公司 融合遥感影像和轨迹数据的分歧合流路口生成方法及装置
CN114998744A (zh) * 2022-07-18 2022-09-02 中国农业大学 基于运动与视觉双特征融合的农机轨迹田路分割方法

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067556B (zh) * 2020-08-05 2023-03-14 北京万集科技股份有限公司 环境感知方法、装置、服务器和可读存储介质
CN112653997A (zh) * 2020-12-29 2021-04-13 西安九索数据技术股份有限公司 一种基于基站序列的位置轨迹计算方法
CN112785466A (zh) * 2020-12-31 2021-05-11 科大讯飞股份有限公司 一种硬件的ai赋能方法、装置、存储介质及设备
CN112801059B (zh) * 2021-04-07 2021-07-20 广东众聚人工智能科技有限公司 图卷积网络系统和基于图卷积网络系统的3d物体检测方法
CN114312831B (zh) * 2021-12-16 2023-10-03 浙江零跑科技股份有限公司 一种基于空间注意力机制的车辆轨迹预测方法
CN114194213A (zh) * 2021-12-29 2022-03-18 北京三快在线科技有限公司 一种目标物轨迹预测方法、装置、存储介质及电子设备
CN115790606B (zh) * 2023-01-09 2023-06-27 深圳鹏行智能研究有限公司 轨迹预测方法、装置、机器人及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN107168342A (zh) * 2017-07-12 2017-09-15 哈尔滨工大智慧工厂有限公司 一种用于机器人路径规划的行人轨迹预测方法
CN107422730A (zh) * 2017-06-09 2017-12-01 武汉市众向科技有限公司 基于视觉导引的agv运输系统及其驾驶控制方法
WO2018081348A1 (en) * 2016-10-26 2018-05-03 The Charles Stark Draper Laboratory, Inc. Vision-inertial navigation with variable contrast tracking residual
CN108022012A (zh) * 2017-12-01 2018-05-11 兰州大学 基于深度学习的车辆位置预测方法
CN108981726A (zh) * 2018-06-09 2018-12-11 安徽宇锋智能科技有限公司 基于感知定位监测的无人车语义地图建模及构建应用方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102070527B1 (ko) * 2017-06-22 2020-01-28 바이두닷컴 타임즈 테크놀로지(베이징) 컴퍼니 리미티드 자율 주행 차량 교통 예측에서 예측되는 궤적에 대한 평가 프레임 워크
US10656657B2 (en) * 2017-08-08 2020-05-19 Uatc, Llc Object motion prediction and autonomous vehicle control
US11112796B2 (en) * 2017-08-08 2021-09-07 Uatc, Llc Object motion prediction and autonomous vehicle control
US11086317B2 (en) * 2018-03-30 2021-08-10 Intel Corporation Emotional adaptive driving policies for automated driving vehicles
CN108734713A (zh) * 2018-05-18 2018-11-02 大连理工大学 一种基于多特征图的交通图像语义分割方法
KR101951595B1 (ko) * 2018-05-18 2019-02-22 한양대학교 산학협력단 모듈형 순환 신경망 구조 기반 차량 경로 예측 시스템 및 방법
CN108803617B (zh) * 2018-07-10 2020-03-20 深圳大学 轨迹预测方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018081348A1 (en) * 2016-10-26 2018-05-03 The Charles Stark Draper Laboratory, Inc. Vision-inertial navigation with variable contrast tracking residual
CN107144285A (zh) * 2017-05-08 2017-09-08 深圳地平线机器人科技有限公司 位姿信息确定方法、装置和可移动设备
CN107422730A (zh) * 2017-06-09 2017-12-01 武汉市众向科技有限公司 基于视觉导引的agv运输系统及其驾驶控制方法
CN107168342A (zh) * 2017-07-12 2017-09-15 哈尔滨工大智慧工厂有限公司 一种用于机器人路径规划的行人轨迹预测方法
CN108022012A (zh) * 2017-12-01 2018-05-11 兰州大学 基于深度学习的车辆位置预测方法
CN108981726A (zh) * 2018-06-09 2018-12-11 安徽宇锋智能科技有限公司 基于感知定位监测的无人车语义地图建模及构建应用方法

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112712013A (zh) * 2020-12-29 2021-04-27 杭州海康威视数字技术股份有限公司 一种移动轨迹构建方法及装置
CN112712013B (zh) * 2020-12-29 2024-01-05 杭州海康威视数字技术股份有限公司 一种移动轨迹构建方法及装置
CN112699942A (zh) * 2020-12-30 2021-04-23 东软睿驰汽车技术（沈阳）有限公司 一种运营车辆识别方法、装置、设备及存储介质
CN112885079B (zh) * 2021-01-11 2022-11-29 成都语动未来科技有限公司 一种基于全局注意力和状态共享的车辆轨迹预测方法
CN112885079A (zh) * 2021-01-11 2021-06-01 成都语动未来科技有限公司 一种基于全局注意力和状态共享的车辆轨迹预测方法
CN113011323A (zh) * 2021-03-18 2021-06-22 北京百度网讯科技有限公司 通行状态的获取方法、相关装置、路侧设备及云控平台
US11783588B2 (en) 2021-03-18 2023-10-10 Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Method for acquiring traffic state, relevant apparatus, roadside device and cloud control platform
CN113139696A (zh) * 2021-05-11 2021-07-20 深圳大学 一种轨迹预测模型构建方法及轨迹预测方法、装置
CN113139696B (zh) * 2021-05-11 2022-09-20 深圳大学 一种轨迹预测模型构建方法及轨迹预测方法、装置
CN113276858A (zh) * 2021-05-13 2021-08-20 际络科技(上海)有限公司 一种节油驾驶控制方法及装置、计算设备和存储介质
CN113759400A (zh) * 2021-08-04 2021-12-07 江苏怀业信息技术股份有限公司 卫星定位轨迹的平滑化方法和装置
CN113759400B (zh) * 2021-08-04 2024-02-27 江苏怀业信息技术股份有限公司 卫星定位轨迹的平滑化方法和装置
CN113743767A (zh) * 2021-08-27 2021-12-03 广东工业大学 基于时间和安全性的车辆派单方法、系统、计算机及介质
CN113934808A (zh) * 2021-10-22 2022-01-14 广东汇天航空航天科技有限公司 地图数据获取方法、装置及飞行器
CN113934808B (zh) * 2021-10-22 2024-05-28 广东汇天航空航天科技有限公司 地图数据获取方法、装置及飞行器
CN114512052A (zh) * 2021-12-31 2022-05-17 武汉中海庭数据技术有限公司 融合遥感影像和轨迹数据的分歧合流路口生成方法及装置
CN114998744A (zh) * 2022-07-18 2022-09-02 中国农业大学 基于运动与视觉双特征融合的农机轨迹田路分割方法

Also Published As

Publication number Publication date
CN111316286A (zh) 2020-06-19

Similar Documents

Publication Publication Date Title
WO2020191642A1 (zh) 轨迹预测方法及装置、存储介质、驾驶系统与车辆
Chen et al. 3d point cloud processing and learning for autonomous driving: Impacting map creation, localization, and perception
Qiao et al. A survey on 5G/6G, AI, and Robotics
Houston et al. One thousand and one hours: Self-driving motion prediction dataset
Bura et al. An edge based smart parking solution using camera networks and deep learning
US20210004012A1 (en) Goal-Directed Occupancy Prediction for Autonomous Driving
CN112212874B (zh) 车辆轨迹预测方法、装置、电子设备及计算机可读介质
JP2023511765A (ja) 軌道予測方法、装置、機器、記憶媒体およびプログラム
KR20220054358A (ko) 점유 예측 뉴럴 네트워크들
US11074438B2 (en) Disentangling human dynamics for pedestrian locomotion forecasting with noisy supervision
US11993291B2 (en) Neural networks for navigation of autonomous vehicles based upon predicted human intents
Wei et al. Survey of connected automated vehicle perception mode: from autonomy to interaction
Yudin et al. Object detection with deep neural networks for reinforcement learning in the task of autonomous vehicles path planning at the intersection
CN110390240A (zh) 自动驾驶车辆中的车道后处理
US20210303877A1 (en) Systems and methods for augmenting perception data with supplemental information
Liang et al. LineNet: A zoomable CNN for crowdsourced high definition maps modeling in urban environments
CN114792148A (zh) 一种运动轨迹的预测方法及装置
US11556126B2 (en) Online agent predictions using semantic maps
Kanchana et al. Computer vision for autonomous driving
Huu et al. Proposing lane and obstacle detection algorithm using YOLO to control self‐driving cars on advanced networks
CN113378605A (zh) 多源信息融合方法及装置、电子设备和存储介质
Lai-Dang et al. Sensor fusion by spatial encoding for autonomous driving
Tran et al. Pedestrian activity prediction based on semantic segmentation and hybrid of machines
US20210383213A1 (en) Prediction device, prediction method, computer program product, and vehicle control system
Rahman Traffic light detection and V2I communications of an autonomous vehicle with the traffic light for an effective intersection navigation using MAVS simulation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19921824

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19921824

Country of ref document: EP

Kind code of ref document: A1